You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/03/09 03:21:28 UTC
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3795 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3795/
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
No registered leader was found, collection:collection1 slice:shard1
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found, collection:collection1 slice:shard1
at __randomizedtesting.SeedInfo.seed([88034211B1D78084:9E5CC09C688E0B8]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:481)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:295)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:116)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9149 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T1913 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /l_/
[junit4:junit4] 2> 5 T1913 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736
[junit4:junit4] 2> 6 T1913 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 7 T1914 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 10 T1914 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 10 T1914 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 10 T1914 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 11 T1914 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 11 T1914 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 108 T1913 oasc.ZkTestServer.run start zk server on port:12121
[junit4:junit4] 2> 108 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@617986b0
[junit4:junit4] 2> 109 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 109 T1919 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 110 T1919 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 110 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12122
[junit4:junit4] 2> 111 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12122
[junit4:junit4] 2> 111 T1917 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 113 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160000 with negotiated timeout 10000 for client /140.211.11.196:12122
[junit4:junit4] 2> 113 T1919 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160000, negotiated timeout = 10000
[junit4:junit4] 2> 114 T1920 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@617986b0 name:ZooKeeperConnection Watcher:127.0.0.1:12121 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 114 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 114 T1913 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 117 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160000
[junit4:junit4] 2> 123 T1920 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 124 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12122 which had sessionid 0x13d4ceb7f160000
[junit4:junit4] 2> 123 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160000 closed
[junit4:junit4] 2> 124 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@11d3d4ff
[junit4:junit4] 2> 125 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 125 T1921 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 126 T1921 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 126 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12123
[junit4:junit4] 2> 126 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12123
[junit4:junit4] 2> 128 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160001 with negotiated timeout 10000 for client /140.211.11.196:12123
[junit4:junit4] 2> 128 T1921 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160001, negotiated timeout = 10000
[junit4:junit4] 2> 128 T1922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11d3d4ff name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 128 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 129 T1913 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 133 T1913 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 136 T1913 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 138 T1913 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 146 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 147 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 161 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 161 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 265 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 266 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 269 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 270 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 273 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 274 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 282 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 282 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 285 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 286 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 289 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 290 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 293 T1913 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 294 T1913 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 301 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160001
[junit4:junit4] 2> 302 T1922 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 302 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12123 which had sessionid 0x13d4ceb7f160001
[junit4:junit4] 2> 302 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160001 closed
[junit4:junit4] 2> 474 T1913 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 480 T1913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12124
[junit4:junit4] 2> 481 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 481 T1913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 482 T1913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033
[junit4:junit4] 2> 482 T1913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/solr.xml
[junit4:junit4] 2> 483 T1913 oasc.CoreContainer.<init> New CoreContainer 337368642
[junit4:junit4] 2> 483 T1913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/'
[junit4:junit4] 2> 484 T1913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/'
[junit4:junit4] 2> 544 T1913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 545 T1913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 545 T1913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 546 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 546 T1913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 547 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 547 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 548 T1913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 548 T1913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 549 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 571 T1913 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 594 T1913 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:12121/solr
[junit4:junit4] 2> 595 T1913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 595 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@58c6c18a
[junit4:junit4] 2> 596 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 597 T1932 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 598 T1932 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 598 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12125
[junit4:junit4] 2> 598 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12125
[junit4:junit4] 2> 614 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160002 with negotiated timeout 20000 for client /140.211.11.196:12125
[junit4:junit4] 2> 614 T1932 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160002, negotiated timeout = 20000
[junit4:junit4] 2> 614 T1933 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@58c6c18a name:ZooKeeperConnection Watcher:127.0.0.1:12121 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 614 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 616 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160002
[junit4:junit4] 2> 640 T1933 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 640 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12125 which had sessionid 0x13d4ceb7f160002
[junit4:junit4] 2> 640 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160002 closed
[junit4:junit4] 2> 640 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 653 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3851a014
[junit4:junit4] 2> 654 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 654 T1934 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 655 T1934 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 655 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12126
[junit4:junit4] 2> 655 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12126
[junit4:junit4] 2> 657 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160003 with negotiated timeout 20000 for client /140.211.11.196:12126
[junit4:junit4] 2> 657 T1934 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160003, negotiated timeout = 20000
[junit4:junit4] 2> 657 T1935 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3851a014 name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 657 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 678 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 680 T1913 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 717 T1913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12124_l_
[junit4:junit4] 2> 718 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:12124_l_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:12124_l_
[junit4:junit4] 2> 757 T1913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12124_l_
[junit4:junit4] 2> 785 T1913 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 841 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 843 T1913 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 880 T1913 oasc.Overseer.start Overseer (id=89312141950451715-127.0.0.1:12124_l_-n_0000000000) starting
[junit4:junit4] 2> 881 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 882 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 888 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 890 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 900 T1937 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 900 T1913 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 910 T1913 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 913 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 915 T1936 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 919 T1938 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/collection1
[junit4:junit4] 2> 920 T1938 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 920 T1938 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 920 T1938 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 922 T1938 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/collection1/'
[junit4:junit4] 2> 923 T1938 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/collection1/lib/README' to classloader
[junit4:junit4] 2> 923 T1938 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 973 T1938 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1031 T1938 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1132 T1938 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 1138 T1938 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1764 T1938 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1775 T1938 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1780 T1938 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1793 T1938 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1798 T1938 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1803 T1938 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1804 T1938 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1804 T1938 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1805 T1938 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1806 T1938 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1806 T1938 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1806 T1938 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362795135033/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data/
[junit4:junit4] 2> 1806 T1938 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bbfdaa4
[junit4:junit4] 2> 1807 T1938 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1808 T1938 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data forceNew: false
[junit4:junit4] 2> 1808 T1938 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data/index/
[junit4:junit4] 2> 1809 T1938 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1809 T1938 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data/index forceNew: false
[junit4:junit4] 2> 1812 T1938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1813 T1938 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1815 T1938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1816 T1938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1816 T1938 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1817 T1938 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1817 T1938 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1818 T1938 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1818 T1938 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1818 T1938 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1819 T1938 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1832 T1938 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1838 T1938 oass.SolrIndexSearcher.<init> Opening Searcher@1cbc622e main
[junit4:junit4] 2> 1838 T1938 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/control/data/tlog
[junit4:junit4] 2> 1839 T1938 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1839 T1938 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1842 T1939 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cbc622e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1844 T1938 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1845 T1938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1845 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 2418 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2419 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:12124_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12124/l_"}
[junit4:junit4] 2> 2419 T1936 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2419 T1936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 2420 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 2440 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2889 T1938 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2889 T1938 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12124/l_ collection:control_collection shard:shard1
[junit4:junit4] 2> 2890 T1938 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2906 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2907 T1938 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2907 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2918 T1938 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2918 T1938 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2918 T1938 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12124/l_/collection1/
[junit4:junit4] 2> 2918 T1938 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2919 T1938 oasc.SyncStrategy.syncToMe http://127.0.0.1:12124/l_/collection1/ has no replicas
[junit4:junit4] 2> 2919 T1938 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12124/l_/collection1/
[junit4:junit4] 2> 2919 T1938 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2923 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3951 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3975 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4003 T1938 oasc.ZkController.register We are http://127.0.0.1:12124/l_/collection1/ and leader is http://127.0.0.1:12124/l_/collection1/
[junit4:junit4] 2> 4003 T1938 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12124/l_
[junit4:junit4] 2> 4003 T1938 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 4003 T1938 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4004 T1938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4005 T1938 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4006 T1913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 4007 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4008 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4022 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 4023 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4483d37f
[junit4:junit4] 2> 4024 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4024 T1941 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4025 T1941 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 4025 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12127
[junit4:junit4] 2> 4025 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12127
[junit4:junit4] 2> 4026 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160004 with negotiated timeout 10000 for client /140.211.11.196:12127
[junit4:junit4] 2> 4026 T1941 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160004, negotiated timeout = 10000
[junit4:junit4] 2> 4027 T1942 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4483d37f name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4027 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4029 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4031 T1913 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4222 T1913 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4225 T1913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12128
[junit4:junit4] 2> 4226 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4227 T1913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4227 T1913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766
[junit4:junit4] 2> 4228 T1913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/solr.xml
[junit4:junit4] 2> 4228 T1913 oasc.CoreContainer.<init> New CoreContainer 1518489380
[junit4:junit4] 2> 4229 T1913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/'
[junit4:junit4] 2> 4229 T1913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/'
[junit4:junit4] 2> 4292 T1913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4293 T1913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4293 T1913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4294 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4294 T1913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4295 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4295 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4296 T1913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4296 T1913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4297 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4317 T1913 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4336 T1913 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:12121/solr
[junit4:junit4] 2> 4337 T1913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4337 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@11a9804
[junit4:junit4] 2> 4338 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4339 T1952 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4340 T1952 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 4340 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12129
[junit4:junit4] 2> 4340 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12129
[junit4:junit4] 2> 4342 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160005 with negotiated timeout 20000 for client /140.211.11.196:12129
[junit4:junit4] 2> 4342 T1952 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160005, negotiated timeout = 20000
[junit4:junit4] 2> 4343 T1953 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11a9804 name:ZooKeeperConnection Watcher:127.0.0.1:12121 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4343 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4344 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160005
[junit4:junit4] 2> 4356 T1953 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4356 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12129 which had sessionid 0x13d4ceb7f160005
[junit4:junit4] 2> 4356 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160005 closed
[junit4:junit4] 2> 4357 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4367 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b3f649d
[junit4:junit4] 2> 4368 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4368 T1954 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4369 T1954 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 4369 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12130
[junit4:junit4] 2> 4369 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12130
[junit4:junit4] 2> 4371 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160006 with negotiated timeout 20000 for client /140.211.11.196:12130
[junit4:junit4] 2> 4371 T1954 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160006, negotiated timeout = 20000
[junit4:junit4] 2> 4371 T1955 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b3f649d name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4371 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4372 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4374 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4382 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5385 T1913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12128_l_
[junit4:junit4] 2> 5386 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:12128_l_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:12128_l_
[junit4:junit4] 2> 5387 T1913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12128_l_
[junit4:junit4] 2> 5391 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 5391 T1942 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5391 T1955 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5392 T1935 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5399 T1956 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/collection1
[junit4:junit4] 2> 5399 T1956 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 5400 T1956 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 5400 T1956 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 5401 T1956 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/collection1/'
[junit4:junit4] 2> 5403 T1956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/collection1/lib/README' to classloader
[junit4:junit4] 2> 5403 T1956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 5453 T1956 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 5489 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 5489 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:12124_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12124/l_"}
[junit4:junit4] 2> 5502 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5502 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5502 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5512 T1956 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5613 T1956 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5620 T1956 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6254 T1956 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6265 T1956 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6269 T1956 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6283 T1956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6288 T1956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6293 T1956 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6294 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6295 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6295 T1956 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6296 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6296 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6297 T1956 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1/
[junit4:junit4] 2> 6297 T1956 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bbfdaa4
[junit4:junit4] 2> 6298 T1956 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6298 T1956 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1 forceNew: false
[junit4:junit4] 2> 6299 T1956 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1/index/
[junit4:junit4] 2> 6299 T1956 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6300 T1956 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1/index forceNew: false
[junit4:junit4] 2> 6302 T1956 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6303 T1956 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6306 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6306 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6307 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6307 T1956 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6308 T1956 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6308 T1956 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6309 T1956 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6309 T1956 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6310 T1956 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6322 T1956 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6329 T1956 oass.SolrIndexSearcher.<init> Opening Searcher@63ae555a main
[junit4:junit4] 2> 6329 T1956 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty1/tlog
[junit4:junit4] 2> 6330 T1956 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6330 T1956 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6335 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@63ae555a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6337 T1956 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 6337 T1956 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 7007 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7008 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12128_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12128/l_"}
[junit4:junit4] 2> 7008 T1936 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 7009 T1936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 7052 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7052 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7052 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7365 T1956 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7365 T1956 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12128/l_ collection:collection1 shard:shard1
[junit4:junit4] 2> 7366 T1956 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7382 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7383 T1956 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7383 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7394 T1956 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7394 T1956 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7395 T1956 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12128/l_/collection1/
[junit4:junit4] 2> 7395 T1956 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7395 T1956 oasc.SyncStrategy.syncToMe http://127.0.0.1:12128/l_/collection1/ has no replicas
[junit4:junit4] 2> 7395 T1956 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12128/l_/collection1/
[junit4:junit4] 2> 7396 T1956 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7400 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8557 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8578 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8578 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8578 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8583 T1956 oasc.ZkController.register We are http://127.0.0.1:12128/l_/collection1/ and leader is http://127.0.0.1:12128/l_/collection1/
[junit4:junit4] 2> 8583 T1956 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12128/l_
[junit4:junit4] 2> 8583 T1956 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8584 T1956 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8584 T1956 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8592 T1956 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8593 T1913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 8594 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8594 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8799 T1913 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8803 T1913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12142
[junit4:junit4] 2> 8804 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8804 T1913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8805 T1913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338
[junit4:junit4] 2> 8805 T1913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/solr.xml
[junit4:junit4] 2> 8806 T1913 oasc.CoreContainer.<init> New CoreContainer 1941417921
[junit4:junit4] 2> 8806 T1913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/'
[junit4:junit4] 2> 8807 T1913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/'
[junit4:junit4] 2> 8877 T1913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8877 T1913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8878 T1913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8878 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8879 T1913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8879 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8880 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8880 T1913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8881 T1913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8881 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8902 T1913 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8921 T1913 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:12121/solr
[junit4:junit4] 2> 8922 T1913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8923 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bc72f4a
[junit4:junit4] 2> 8924 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8924 T1968 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8926 T1968 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 8926 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12143
[junit4:junit4] 2> 8926 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12143
[junit4:junit4] 2> 8928 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160007 with negotiated timeout 20000 for client /140.211.11.196:12143
[junit4:junit4] 2> 8928 T1968 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160007, negotiated timeout = 20000
[junit4:junit4] 2> 8929 T1969 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bc72f4a name:ZooKeeperConnection Watcher:127.0.0.1:12121 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8929 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8930 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160007
[junit4:junit4] 2> 8942 T1969 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8942 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12143 which had sessionid 0x13d4ceb7f160007
[junit4:junit4] 2> 8942 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160007 closed
[junit4:junit4] 2> 8943 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8953 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4a5fcf47
[junit4:junit4] 2> 8954 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8954 T1970 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8955 T1970 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 8955 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12144
[junit4:junit4] 2> 8955 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12144
[junit4:junit4] 2> 8957 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160008 with negotiated timeout 20000 for client /140.211.11.196:12144
[junit4:junit4] 2> 8957 T1970 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160008, negotiated timeout = 20000
[junit4:junit4] 2> 8957 T1971 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a5fcf47 name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8957 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8958 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8960 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8968 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9971 T1913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12142_l_
[junit4:junit4] 2> 9972 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:12142_l_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:12142_l_
[junit4:junit4] 2> 9975 T1913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12142_l_
[junit4:junit4] 2> 9978 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9978 T1942 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9978 T1955 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9978 T1971 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9978 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9979 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9979 T1935 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9992 T1972 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/collection1
[junit4:junit4] 2> 9992 T1972 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9993 T1972 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9993 T1972 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9994 T1972 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/collection1/'
[junit4:junit4] 2> 9996 T1972 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9996 T1972 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/collection1/lib/README' to classloader
[junit4:junit4] 2> 10046 T1972 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 10093 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10094 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12128_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12128/l_"}
[junit4:junit4] 2> 10104 T1972 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 10107 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10107 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10107 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10108 T1972 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 10107 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10114 T1972 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 10751 T1972 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 10763 T1972 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 10767 T1972 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 10781 T1972 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10786 T1972 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10791 T1972 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10792 T1972 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10793 T1972 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10793 T1972 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10794 T1972 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10795 T1972 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10795 T1972 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362795143338/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2/
[junit4:junit4] 2> 10795 T1972 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bbfdaa4
[junit4:junit4] 2> 10796 T1972 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10797 T1972 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2 forceNew: false
[junit4:junit4] 2> 10797 T1972 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2/index/
[junit4:junit4] 2> 10798 T1972 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 10798 T1972 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2/index forceNew: false
[junit4:junit4] 2> 10801 T1972 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 10801 T1972 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 10804 T1972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 10805 T1972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 10805 T1972 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 10806 T1972 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 10807 T1972 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 10807 T1972 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 10807 T1972 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 10808 T1972 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 10808 T1972 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 10822 T1972 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 10828 T1972 oass.SolrIndexSearcher.<init> Opening Searcher@5da19ba2 main
[junit4:junit4] 2> 10828 T1972 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty2/tlog
[junit4:junit4] 2> 10829 T1972 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 10829 T1972 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 10834 T1973 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5da19ba2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 10836 T1972 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 10836 T1972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11612 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11613 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12142_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12142/l_"}
[junit4:junit4] 2> 11613 T1936 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11614 T1936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11633 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11633 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11633 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11633 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11839 T1972 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11839 T1972 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12142/l_ collection:collection1 shard:shard2
[junit4:junit4] 2> 11840 T1972 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 11854 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:delete cxid:0x47 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 11855 T1972 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11855 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:create cxid:0x48 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11857 T1972 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11858 T1972 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11858 T1972 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12142/l_/collection1/
[junit4:junit4] 2> 11858 T1972 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11858 T1972 oasc.SyncStrategy.syncToMe http://127.0.0.1:12142/l_/collection1/ has no replicas
[junit4:junit4] 2> 11859 T1972 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12142/l_/collection1/
[junit4:junit4] 2> 11859 T1972 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 11867 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160008 type:create cxid:0x51 zxid:0x83 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13137 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13158 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13158 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13158 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13158 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13195 T1972 oasc.ZkController.register We are http://127.0.0.1:12142/l_/collection1/ and leader is http://127.0.0.1:12142/l_/collection1/
[junit4:junit4] 2> 13195 T1972 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12142/l_
[junit4:junit4] 2> 13195 T1972 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 13195 T1972 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 13196 T1972 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13197 T1972 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13198 T1913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 13199 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 13200 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 13399 T1913 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 13402 T1913 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:12146
[junit4:junit4] 2> 13403 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 13404 T1913 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 13404 T1913 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944
[junit4:junit4] 2> 13405 T1913 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/solr.xml
[junit4:junit4] 2> 13405 T1913 oasc.CoreContainer.<init> New CoreContainer 1848339858
[junit4:junit4] 2> 13406 T1913 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/'
[junit4:junit4] 2> 13406 T1913 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/'
[junit4:junit4] 2> 13469 T1913 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 13470 T1913 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 13471 T1913 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 13471 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 13472 T1913 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 13472 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 13473 T1913 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 13473 T1913 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 13474 T1913 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 13474 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 13494 T1913 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 13513 T1913 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:12121/solr
[junit4:junit4] 2> 13514 T1913 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 13515 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4588e984
[junit4:junit4] 2> 13516 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13516 T1984 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13517 T1984 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 13517 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12147
[junit4:junit4] 2> 13518 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12147
[junit4:junit4] 2> 13520 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f160009 with negotiated timeout 20000 for client /140.211.11.196:12147
[junit4:junit4] 2> 13520 T1984 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f160009, negotiated timeout = 20000
[junit4:junit4] 2> 13520 T1985 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4588e984 name:ZooKeeperConnection Watcher:127.0.0.1:12121 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13520 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13522 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f160009
[junit4:junit4] 2> 13533 T1985 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 13533 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12147 which had sessionid 0x13d4ceb7f160009
[junit4:junit4] 2> 13533 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f160009 closed
[junit4:junit4] 2> 13534 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 13544 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@18b97ccc
[junit4:junit4] 2> 13545 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13545 T1986 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13546 T1986 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 13546 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12148
[junit4:junit4] 2> 13546 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12148
[junit4:junit4] 2> 13547 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f16000a with negotiated timeout 20000 for client /140.211.11.196:12148
[junit4:junit4] 2> 13547 T1986 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f16000a, negotiated timeout = 20000
[junit4:junit4] 2> 13548 T1987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18b97ccc name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13548 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13549 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f16000a type:create cxid:0x1 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13551 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f16000a type:create cxid:0x2 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13553 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 14556 T1913 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12146_l_
[junit4:junit4] 2> 14557 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f16000a type:delete cxid:0xb zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:12146_l_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:12146_l_
[junit4:junit4] 2> 14558 T1913 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12146_l_
[junit4:junit4] 2> 14561 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14562 T1942 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14562 T1971 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14562 T1987 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14562 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14562 T1955 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14562 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14563 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14563 T1935 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14578 T1988 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/collection1
[junit4:junit4] 2> 14578 T1988 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 14579 T1988 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 14579 T1988 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 14580 T1988 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/collection1/'
[junit4:junit4] 2> 14582 T1988 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 14582 T1988 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/collection1/lib/README' to classloader
[junit4:junit4] 2> 14631 T1988 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 14664 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14665 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12142_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12142/l_"}
[junit4:junit4] 2> 14680 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14680 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14681 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14681 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14680 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14690 T1988 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 14790 T1988 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 14796 T1988 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15418 T1988 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15429 T1988 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15433 T1988 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15448 T1988 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15453 T1988 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15458 T1988 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15459 T1988 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15460 T1988 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15461 T1988 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15462 T1988 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15462 T1988 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15462 T1988 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362795147944/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/
[junit4:junit4] 2> 15463 T1988 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bbfdaa4
[junit4:junit4] 2> 15463 T1988 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15464 T1988 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3 forceNew: false
[junit4:junit4] 2> 15464 T1988 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index/
[junit4:junit4] 2> 15465 T1988 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15465 T1988 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index forceNew: false
[junit4:junit4] 2> 15468 T1988 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15468 T1988 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15471 T1988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15472 T1988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15472 T1988 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15473 T1988 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15473 T1988 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15474 T1988 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15474 T1988 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15475 T1988 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15475 T1988 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15488 T1988 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15494 T1988 oass.SolrIndexSearcher.<init> Opening Searcher@4e501e84 main
[junit4:junit4] 2> 15494 T1988 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/tlog
[junit4:junit4] 2> 15495 T1988 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15496 T1988 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15500 T1989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e501e84 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15502 T1988 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15503 T1988 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16185 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16186 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12146_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12146/l_"}
[junit4:junit4] 2> 16187 T1936 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 16187 T1936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16210 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16210 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16210 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16210 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16210 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16506 T1988 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16506 T1988 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12146/l_ collection:collection1 shard:shard1
[junit4:junit4] 2> 16510 T1988 oasc.ZkController.register We are http://127.0.0.1:12146/l_/collection1/ and leader is http://127.0.0.1:12128/l_/collection1/
[junit4:junit4] 2> 16511 T1988 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12146/l_
[junit4:junit4] 2> 16511 T1988 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16511 T1988 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C98 name=collection1 org.apache.solr.core.SolrCore@5dff2ffe url=http://127.0.0.1:12146/l_/collection1 node=127.0.0.1:12146_l_ C98_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:12146_l_, base_url=http://127.0.0.1:12146/l_}
[junit4:junit4] 2> 16511 T1990 C98 P12146 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16512 T1988 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16513 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16513 T1990 C98 P12146 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16513 T1990 C98 P12146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16514 T1913 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 16514 T1913 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16515 T1990 C98 P12146 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16515 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16536 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16537 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16550 T1946 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/onenodecollectioncore
[junit4:junit4] 2> 16551 T1946 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16552 T1946 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16552 T1946 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16553 T1946 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16554 T1946 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16556 T1946 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16558 T1946 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/onenodecollectioncore/'
[junit4:junit4] 2> 16627 T1946 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16686 T1946 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16787 T1946 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16793 T1946 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17415 T1946 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17426 T1946 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17431 T1946 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17444 T1946 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17449 T1946 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17454 T1946 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17455 T1946 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17457 T1946 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17457 T1946 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17458 T1946 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17459 T1946 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17459 T1946 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362795138766/onenodecollectioncore/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection/
[junit4:junit4] 2> 17459 T1946 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bbfdaa4
[junit4:junit4] 2> 17460 T1946 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17461 T1946 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection forceNew: false
[junit4:junit4] 2> 17462 T1946 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection/index/
[junit4:junit4] 2> 17462 T1946 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17463 T1946 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection/index forceNew: false
[junit4:junit4] 2> 17466 T1946 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7d335694 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f21ffa2)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17466 T1946 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17469 T1946 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17470 T1946 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17471 T1946 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17471 T1946 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17472 T1946 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17472 T1946 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17473 T1946 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17474 T1946 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17474 T1946 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17487 T1946 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17493 T1946 oass.SolrIndexSearcher.<init> Opening Searcher@626b1ff4 main
[junit4:junit4] 2> 17494 T1946 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection/tlog
[junit4:junit4] 2> 17495 T1946 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17495 T1946 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17499 T1992 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@626b1ff4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17501 T1946 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 17715 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17716 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12146_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12146/l_"}
[junit4:junit4] 2> 17719 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:12128_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12128/l_"}
[junit4:junit4] 2> 17720 T1936 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 17720 T1936 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17733 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17733 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17733 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17733 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17733 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18513 T1946 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 18513 T1946 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:12128/l_ collection:onenodecollection shard:shard1
[junit4:junit4] 2> 18514 T1946 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 18539 T1945 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:12146_l__collection1&state=recovering&nodeName=127.0.0.1:12146_l_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 18540 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:delete cxid:0xa1 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 18541 T1946 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 18542 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0xa2 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18544 T1946 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 18545 T1946 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 18545 T1946 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12128/l_/onenodecollectioncore/
[junit4:junit4] 2> 18546 T1946 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 18546 T1946 oasc.SyncStrategy.syncToMe http://127.0.0.1:12128/l_/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 18547 T1946 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12128/l_/onenodecollectioncore/
[junit4:junit4] 2> 18548 T1946 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 18552 T1918 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ceb7f160006 type:create cxid:0xac zxid:0xb2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 19240 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19258 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19259 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19259 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19259 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19258 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19268 T1946 oasc.ZkController.register We are http://127.0.0.1:12128/l_/onenodecollectioncore/ and leader is http://127.0.0.1:12128/l_/onenodecollectioncore/
[junit4:junit4] 2> 19268 T1946 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:12128/l_
[junit4:junit4] 2> 19268 T1946 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 19269 T1946 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 19270 T1946 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19271 T1946 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&roles=none&name=onenodecollectioncore&action=CREATE&collection=onenodecollection&dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362795134736/onenodecollection&wt=javabin&version=2} status=0 QTime=2721
[junit4:junit4] 2> 19272 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 19273 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20275 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C98_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:12146_l_, base_url=http://127.0.0.1:12146/l_}
[junit4:junit4] 2> 20541 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:12128/l_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20541 T1990 C98 P12146 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:12146/l_ START replicas=[http://127.0.0.1:12128/l_/collection1/] nUpdates=100
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20542 T1990 C98 P12146 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:12128/l_/collection1/. core=collection1
[junit4:junit4] 2> 20543 T1990 C98 P12146 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C99 name=collection1 org.apache.solr.core.SolrCore@450592c7 url=http://127.0.0.1:12128/l_/collection1 node=127.0.0.1:12128_l_ C99_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:12128_l_, base_url=http://127.0.0.1:12128/l_, leader=true}
[junit4:junit4] 2> 20552 T1947 C99 P12128 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 20562 T1948 C99 P12128 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20564 T1948 C99 P12128 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20565 T1948 C99 P12128 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20566 T1948 C99 P12128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20566 T1948 C99 P12128 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20567 T1948 C99 P12128 oass.SolrIndexSearcher.<init> Opening Searcher@1ec08d7d realtime
[junit4:junit4] 2> 20567 T1948 C99 P12128 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20567 T1948 C99 P12128 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 20568 T1990 C98 P12146 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20569 T1990 C98 P12146 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20571 T1949 C99 P12128 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20571 T1949 C99 P12128 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 20572 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 20572 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20572 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20574 T1949 C99 P12128 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 20575 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20576 T1990 C98 P12146 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index.20130308181235305 forceNew: false
[junit4:junit4] 2> 20576 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@37df5e23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d36fbe3) fullCopy=false
[junit4:junit4] 2> 20579 T1949 C99 P12128 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 20581 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20582 T1990 C98 P12146 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20582 T1990 C98 P12146 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20584 T1990 C98 P12146 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20584 T1990 C98 P12146 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20585 T1990 C98 P12146 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20585 T1990 C98 P12146 oass.SolrIndexSearcher.<init> Opening Searcher@2d6b8b49 main
[junit4:junit4] 2> 20586 T1989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d6b8b49 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 20587 T1990 C98 P12146 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@37df5e23 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d36fbe3)
[junit4:junit4] 2> 20587 T1990 C98 P12146 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20587 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20587 T1990 C98 P12146 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20588 T1990 C98 P12146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20614 T1990 C98 P12146 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20764 T1936 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20765 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:12128_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12128/l_"}
[junit4:junit4] 2> 20782 T1936 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:12146_l_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:12146/l_"}
[junit4:junit4] 2> 20820 T1942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20820 T1935 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20820 T1971 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20820 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20820 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21277 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21278 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 21279 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C100 name=onenodecollectioncore org.apache.solr.core.SolrCore@683419f9 url=http://127.0.0.1:12128/l_/onenodecollectioncore node=127.0.0.1:12128_l_ C100_STATE=coll:onenodecollection core:onenodecollectioncore props:{shard=shard1, roles=none, state=active, core=onenodecollectioncore, collection=onenodecollection, node_name=127.0.0.1:12128_l_, base_url=http://127.0.0.1:12128/l_, leader=true}
[junit4:junit4] 2> 21299 T1950 C100 P12128 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7d335694 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f21ffa2)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21300 T1950 C100 P12128 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21302 T1950 C100 P12128 /update {wt=javabin&version=2} {add=[1 (1428994293527216128)]} 0 5
[junit4:junit4] 2> 21306 T1950 C100 P12128 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21309 T1950 C100 P12128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7d335694 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f21ffa2)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@7d335694 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f21ffa2)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_MockSep_0.frq, segments_2, _0_Direct_0.pos, _0_Direct_0.tip, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.doc, _0_MockSep_0.pyl, _0_Direct_0.tim, _0.fdt]
[junit4:junit4] 2> 21309 T1950 C100 P12128 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_MockSep_0.frq, segments_2, _0_Direct_0.pos, _0_Direct_0.tip, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.doc, _0_MockSep_0.pyl, _0_Direct_0.tim, _0.fdt]
[junit4:junit4] 2> 21362 T1950 C100 P12128 oass.SolrIndexSearcher.<init> Opening Searcher@3451c668 main
[junit4:junit4] 2> 21362 T1950 C100 P12128 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21363 T1992 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@3451c668 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21364 T1950 C100 P12128 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 21368 T1950 C100 P12128 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21369 T1913 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 21381 T1950 C100 P12128 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C101 name=collection1 org.apache.solr.core.SolrCore@1cc0dce3 url=http://127.0.0.1:12124/l_/collection1 node=127.0.0.1:12124_l_ C101_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:12124_l_, base_url=http://127.0.0.1:12124/l_, leader=true}
[junit4:junit4] 2> 21391 T1925 C101 P12124 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21392 T1925 C101 P12124 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21395 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[1 (1428994293622636544)]} 0 6
[junit4:junit4] 2> C98_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:12146_l_, base_url=http://127.0.0.1:12146/l_}
[junit4:junit4] 2> 21426 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1428994293647802368)]} 0 3
[junit4:junit4] 2> 21427 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1428994293647802368)]} 0 14
[junit4:junit4] 2> ASYNC NEW_CORE C102 name=collection1 org.apache.solr.core.SolrCore@4fb3c0ab url=http://127.0.0.1:12142/l_/collection1 node=127.0.0.1:12142_l_ C102_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:12142_l_, base_url=http://127.0.0.1:12142/l_, leader=true}
[junit4:junit4] 2> 21428 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[1]} 0 17
[junit4:junit4] 2> 21430 T1925 C101 P12124 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21435 T1925 C101 P12124 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, segments_2, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt]
[junit4:junit4] 2> 21435 T1925 C101 P12124 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, segments_2, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt]
[junit4:junit4] 2> 21438 T1925 C101 P12124 oass.SolrIndexSearcher.<init> Opening Searcher@76130e87 main
[junit4:junit4] 2> 21439 T1925 C101 P12124 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21439 T1939 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76130e87 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21440 T1925 C101 P12124 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 21441 T1945 C99 P12128 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21446 T1945 C99 P12128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 21447 T1945 C99 P12128 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 21449 T1945 C99 P12128 oass.SolrIndexSearcher.<init> Opening Searcher@5149f23d main
[junit4:junit4] 2> 21450 T1945 C99 P12128 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21451 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5149f23d main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21451 T1945 C99 P12128 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:12146/l_/collection1/, StdNode: http://127.0.0.1:12142/l_/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 21453 T1964 C102 P12142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21454 T1977 C98 P12146 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21455 T1964 C102 P12142 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21456 T1964 C102 P12142 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21457 T1964 C102 P12142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 21457 T1964 C102 P12142 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 21458 T1964 C102 P12142 oass.SolrIndexSearcher.<init> Opening Searcher@524729c6 main
[junit4:junit4] 2> 21458 T1964 C102 P12142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21459 T1973 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@524729c6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 21459 T1977 C98 P12146 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 21460 T1964 C102 P12142 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 21460 T1977 C98 P12146 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 21462 T1977 C98 P12146 oass.SolrIndexSearcher.<init> Opening Searcher@236d7114 main
[junit4:junit4] 2> 21463 T1977 C98 P12146 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21464 T1989 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@236d7114 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21464 T1977 C98 P12146 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 21465 T1945 C99 P12128 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 21465 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21466 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21467 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 21470 T1925 C101 P12124 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21471 T1913 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:12121/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@661aea37
[junit4:junit4] 2> 21472 T1913 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 21472 T1999 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:12121. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 21473 T1999 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:12121, initiating session
[junit4:junit4] 2> 21473 T1915 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:56483
[junit4:junit4] 2> 21473 T1915 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:56483
[junit4:junit4] 2> 21496 T1917 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ceb7f16000b with negotiated timeout 10000 for client /140.211.11.196:56483
[junit4:junit4] 2> 21496 T1999 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:12121, sessionid = 0x13d4ceb7f16000b, negotiated timeout = 10000
[junit4:junit4] 2> 21496 T2000 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@661aea37 name:ZooKeeperConnection Watcher:127.0.0.1:12121/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 21496 T1913 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 21498 T1913 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 21500 T1918 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ceb7f16000b
[junit4:junit4] 2> 21501 T2000 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21501 T1915 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:56483 which had sessionid 0x13d4ceb7f16000b
[junit4:junit4] 2> 21501 T1913 oaz.ZooKeeper.close Session: 0x13d4ceb7f16000b closed
[junit4:junit4] 2> 21503 T1948 C99 P12128 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21505 T1965 C102 P12142 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21509 T1966 C102 P12142 REQ /select {NOW=1362795156237&shard.url=127.0.0.1:12142/l_/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21509 T2003 C99 P12128 REQ /select {NOW=1362795156237&shard.url=127.0.0.1:12128/l_/collection1/|127.0.0.1:12146/l_/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21512 T2003 C99 P12128 REQ /select {NOW=1362795156237&shard.url=127.0.0.1:12128/l_/collection1/|127.0.0.1:12146/l_/collection1/&ids=1&q=*:*&distrib=false&wt=javabin&isShard=true&version=2} status=0 QTime=0
[junit4:junit4] 2> 21513 T1978 C98 P12146 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=6
[junit4:junit4] 2> 21519 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[2 (1428994293755805696)]} 0 3
[junit4:junit4] 2> 21525 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1428994293763145728)]} 0 2
[junit4:junit4] 2> 21526 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[2]} 0 5
[junit4:junit4] 2> 21530 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[3 (1428994293769437184)]} 0 1
[junit4:junit4] 2> 21536 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1428994293775728640)]} 0 1
[junit4:junit4] 2> 21537 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[3]} 0 4
[junit4:junit4] 2> 21540 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[4 (1428994293779922944)]} 0 1
[junit4:junit4] 2> 21551 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1428994293786214400)]} 0 2
[junit4:junit4] 2> 21552 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1428994293786214400)]} 0 7
[junit4:junit4] 2> 21552 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[4]} 0 9
[junit4:junit4] 2> 21556 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[5 (1428994293796700160)]} 0 1
[junit4:junit4] 2> 21561 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[5 (1428994293800894464)]} 0 2
[junit4:junit4] 2> 21564 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[6 (1428994293805088768)]} 0 1
[junit4:junit4] 2> 21570 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1428994293811380224)]} 0 1
[junit4:junit4] 2> 21571 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[6]} 0 4
[junit4:junit4] 2> 21574 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[7 (1428994293815574528)]} 0 1
[junit4:junit4] 2> 21578 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[7 (1428994293819768832)]} 0 1
[junit4:junit4] 2> 21582 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[8 (1428994293823963136)]} 0 1
[junit4:junit4] 2> 21591 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1428994293830254592)]} 0 1
[junit4:junit4] 2> 21592 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1428994293830254592)]} 0 5
[junit4:junit4] 2> 21592 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[8]} 0 7
[junit4:junit4] 2> 21596 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[9 (1428994293838643200)]} 0 1
[junit4:junit4] 2> 21600 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[9 (1428994293842837504)]} 0 1
[junit4:junit4] 2> 21604 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[10 (1428994293847031808)]} 0 1
[junit4:junit4] 2> 21612 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1428994293851226112)]} 0 1
[junit4:junit4] 2> 21613 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[10 (1428994293851226112)]} 0 6
[junit4:junit4] 2> 21617 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[11 (1428994293860663296)]} 0 1
[junit4:junit4] 2> 21625 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1428994293864857600)]} 0 1
[junit4:junit4] 2> 21626 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[11 (1428994293864857600)]} 0 6
[junit4:junit4] 2> 21630 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[12 (1428994293874294784)]} 0 1
[junit4:junit4] 2> 21639 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1428994293880586240)]} 0 1
[junit4:junit4] 2> 21640 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1428994293880586240)]} 0 5
[junit4:junit4] 2> 21640 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[12]} 0 7
[junit4:junit4] 2> 21644 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[13 (1428994293888974848)]} 0 1
[junit4:junit4] 2> 21651 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1428994293893169152)]} 0 1
[junit4:junit4] 2> 21652 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[13 (1428994293893169152)]} 0 5
[junit4:junit4] 2> 21655 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[14 (1428994293900509184)]} 0 1
[junit4:junit4] 2> 21662 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1428994293904703488)]} 0 1
[junit4:junit4] 2> 21663 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[14 (1428994293904703488)]} 0 5
[junit4:junit4] 2> 21666 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[15 (1428994293912043520)]} 0 1
[junit4:junit4] 2> 21674 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1428994293917286400)]} 0 1
[junit4:junit4] 2> 21674 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1428994293917286400)]} 0 4
[junit4:junit4] 2> 21675 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[15]} 0 7
[junit4:junit4] 2> 21679 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[16 (1428994293925675008)]} 0 1
[junit4:junit4] 2> 21688 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1428994293931966464)]} 0 1
[junit4:junit4] 2> 21688 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1428994293931966464)]} 0 4
[junit4:junit4] 2> 21689 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[16]} 0 7
[junit4:junit4] 2> 21694 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[17 (1428994293940355072)]} 0 2
[junit4:junit4] 2> 21701 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1428994293947695104)]} 0 2
[junit4:junit4] 2> 21702 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[17]} 0 5
[junit4:junit4] 2> 21706 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[100 (1428994293953986560)]} 0 1
[junit4:junit4] 2> 21714 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[100 (1428994293959229440)]} 0 1
[junit4:junit4] 2> 21714 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[100 (1428994293959229440)]} 0 4
[junit4:junit4] 2> 21715 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[100]} 0 7
[junit4:junit4] 2> 21719 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[101 (1428994293967618048)]} 0 1
[junit4:junit4] 2> 21727 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[101 (1428994293972860928)]} 0 1
[junit4:junit4] 2> 21727 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[101 (1428994293972860928)]} 0 4
[junit4:junit4] 2> 21728 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[101]} 0 7
[junit4:junit4] 2> 21732 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[102 (1428994293981249536)]} 0 1
[junit4:junit4] 2> 21740 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[102 (1428994293986492416)]} 0 1
[junit4:junit4] 2> 21740 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[102 (1428994293986492416)]} 0 4
[junit4:junit4] 2> 21741 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[102]} 0 7
[junit4:junit4] 2> 21745 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[103 (1428994293994881024)]} 0 1
[junit4:junit4] 2> 21748 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[103 (1428994293998026752)]} 0 1
[junit4:junit4] 2> 21752 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[104 (1428994294002221056)]} 0 1
[junit4:junit4] 2> 21757 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[104 (1428994294007463936)]} 0 1
[junit4:junit4] 2> 21757 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[104]} 0 3
[junit4:junit4] 2> 21761 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[105 (1428994294011658240)]} 0 1
[junit4:junit4] 2> 21764 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[105 (1428994294014803968)]} 0 1
[junit4:junit4] 2> 21768 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[106 (1428994294018998272)]} 0 1
[junit4:junit4] 2> 21773 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[106 (1428994294024241152)]} 0 1
[junit4:junit4] 2> 21773 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[106]} 0 3
[junit4:junit4] 2> 21777 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[107 (1428994294028435456)]} 0 1
[junit4:junit4] 2> 21785 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[107 (1428994294033678336)]} 0 1
[junit4:junit4] 2> 21785 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[107 (1428994294033678336)]} 0 4
[junit4:junit4] 2> 21786 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[107]} 0 7
[junit4:junit4] 2> 21790 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[108 (1428994294042066944)]} 0 1
[junit4:junit4] 2> 21796 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[108 (1428994294045212672)]} 0 1
[junit4:junit4] 2> 21796 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[108 (1428994294045212672)]} 0 4
[junit4:junit4] 2> 21800 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[109 (1428994294052552704)]} 0 1
[junit4:junit4] 2> 21806 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[109 (1428994294055698432)]} 0 1
[junit4:junit4] 2> 21806 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[109 (1428994294055698432)]} 0 4
[junit4:junit4] 2> 21810 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[110 (1428994294063038464)]} 0 1
[junit4:junit4] 2> 21816 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[110 (1428994294066184192)]} 0 1
[junit4:junit4] 2> 21816 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[110 (1428994294066184192)]} 0 4
[junit4:junit4] 2> 21820 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[111 (1428994294073524224)]} 0 1
[junit4:junit4] 2> 21828 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[111 (1428994294078767104)]} 0 1
[junit4:junit4] 2> 21828 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[111 (1428994294078767104)]} 0 4
[junit4:junit4] 2> 21829 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[111]} 0 7
[junit4:junit4] 2> 21833 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[112 (1428994294087155712)]} 0 1
[junit4:junit4] 2> 21838 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[112 (1428994294092398592)]} 0 1
[junit4:junit4] 2> 21838 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[112]} 0 3
[junit4:junit4] 2> 21842 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[113 (1428994294096592896)]} 0 1
[junit4:junit4] 2> 21850 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[113 (1428994294101835776)]} 0 1
[junit4:junit4] 2> 21850 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[113 (1428994294101835776)]} 0 4
[junit4:junit4] 2> 21851 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[113]} 0 7
[junit4:junit4] 2> 21855 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[114 (1428994294110224384)]} 0 1
[junit4:junit4] 2> 21863 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[114 (1428994294115467264)]} 0 1
[junit4:junit4] 2> 21863 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[114 (1428994294115467264)]} 0 4
[junit4:junit4] 2> 21864 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[114]} 0 7
[junit4:junit4] 2> 21868 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[115 (1428994294124904448)]} 0 1
[junit4:junit4] 2> 21872 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[115 (1428994294128050176)]} 0 1
[junit4:junit4] 2> 21876 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[116 (1428994294132244480)]} 0 1
[junit4:junit4] 2> 21881 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[116 (1428994294137487360)]} 0 1
[junit4:junit4] 2> 21881 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[116]} 0 3
[junit4:junit4] 2> 21885 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[117 (1428994294141681664)]} 0 1
[junit4:junit4] 2> 21890 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[117 (1428994294146924544)]} 0 1
[junit4:junit4] 2> 21890 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[117]} 0 3
[junit4:junit4] 2> 21894 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[118 (1428994294151118848)]} 0 1
[junit4:junit4] 2> 21902 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[118 (1428994294156361728)]} 0 1
[junit4:junit4] 2> 21902 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[118 (1428994294156361728)]} 0 4
[junit4:junit4] 2> 21903 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[118]} 0 7
[junit4:junit4] 2> 21907 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[119 (1428994294164750336)]} 0 1
[junit4:junit4] 2> 21912 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[119 (1428994294169993216)]} 0 1
[junit4:junit4] 2> 21912 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[119]} 0 3
[junit4:junit4] 2> 21916 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[120 (1428994294174187520)]} 0 1
[junit4:junit4] 2> 21919 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[120 (1428994294177333248)]} 0 1
[junit4:junit4] 2> 21923 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[121 (1428994294181527552)]} 0 1
[junit4:junit4] 2> 21926 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[121 (1428994294184673280)]} 0 1
[junit4:junit4] 2> 21930 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[122 (1428994294188867584)]} 0 1
[junit4:junit4] 2> 21938 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[122 (1428994294194110464)]} 0 1
[junit4:junit4] 2> 21938 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[122 (1428994294194110464)]} 0 4
[junit4:junit4] 2> 21939 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[122]} 0 7
[junit4:junit4] 2> 21943 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[123 (1428994294202499072)]} 0 1
[junit4:junit4] 2> 21951 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[123 (1428994294207741952)]} 0 1
[junit4:junit4] 2> 21951 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[123 (1428994294207741952)]} 0 4
[junit4:junit4] 2> 21952 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[123]} 0 7
[junit4:junit4] 2> 21956 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[124 (1428994294216130560)]} 0 1
[junit4:junit4] 2> 21964 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[124 (1428994294221373440)]} 0 1
[junit4:junit4] 2> 21964 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[124 (1428994294221373440)]} 0 4
[junit4:junit4] 2> 21965 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[124]} 0 7
[junit4:junit4] 2> 21969 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[125 (1428994294229762048)]} 0 1
[junit4:junit4] 2> 21977 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[125 (1428994294235004928)]} 0 1
[junit4:junit4] 2> 21977 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[125 (1428994294235004928)]} 0 4
[junit4:junit4] 2> 21978 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[125]} 0 7
[junit4:junit4] 2> 21982 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[126 (1428994294243393536)]} 0 1
[junit4:junit4] 2> 21985 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[126 (1428994294246539264)]} 0 1
[junit4:junit4] 2> 21989 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[127 (1428994294250733568)]} 0 1
[junit4:junit4] 2> 21992 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[127 (1428994294253879296)]} 0 1
[junit4:junit4] 2> 21996 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[128 (1428994294258073600)]} 0 1
[junit4:junit4] 2> 21999 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[128 (1428994294261219328)]} 0 1
[junit4:junit4] 2> 22003 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[129 (1428994294265413632)]} 0 1
[junit4:junit4] 2> 22010 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[129 (1428994294269607936)]} 0 1
[junit4:junit4] 2> 22010 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[129 (1428994294269607936)]} 0 4
[junit4:junit4] 2> 22014 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[130 (1428994294276947968)]} 0 1
[junit4:junit4] 2> 22017 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[130 (1428994294280093696)]} 0 1
[junit4:junit4] 2> 22021 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[131 (1428994294284288000)]} 0 1
[junit4:junit4] 2> 22024 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[131 (1428994294287433728)]} 0 1
[junit4:junit4] 2> 22028 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[132 (1428994294291628032)]} 0 1
[junit4:junit4] 2> 22036 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[132 (1428994294296870912)]} 0 1
[junit4:junit4] 2> 22036 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[132 (1428994294296870912)]} 0 4
[junit4:junit4] 2> 22037 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[132]} 0 7
[junit4:junit4] 2> 22041 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[133 (1428994294305259520)]} 0 1
[junit4:junit4] 2> 22049 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[133 (1428994294310502400)]} 0 1
[junit4:junit4] 2> 22049 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[133 (1428994294310502400)]} 0 4
[junit4:junit4] 2> 22050 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[133]} 0 7
[junit4:junit4] 2> 22054 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[134 (1428994294318891008)]} 0 1
[junit4:junit4] 2> 22057 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[134 (1428994294322036736)]} 0 1
[junit4:junit4] 2> 22060 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[135 (1428994294325182464)]} 0 1
[junit4:junit4] 2> 22069 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[135 (1428994294331473920)]} 0 1
[junit4:junit4] 2> 22069 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[135 (1428994294331473920)]} 0 4
[junit4:junit4] 2> 22070 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[135]} 0 7
[junit4:junit4] 2> 22074 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[136 (1428994294339862528)]} 0 1
[junit4:junit4] 2> 22077 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[136 (1428994294343008256)]} 0 1
[junit4:junit4] 2> 22081 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[137 (1428994294347202560)]} 0 1
[junit4:junit4] 2> 22086 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[137 (1428994294352445440)]} 0 1
[junit4:junit4] 2> 22086 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[137]} 0 3
[junit4:junit4] 2> 22090 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[138 (1428994294356639744)]} 0 1
[junit4:junit4] 2> 22096 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[138 (1428994294359785472)]} 0 1
[junit4:junit4] 2> 22096 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[138 (1428994294359785472)]} 0 4
[junit4:junit4] 2> 22100 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[139 (1428994294367125504)]} 0 1
[junit4:junit4] 2> 22105 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[139 (1428994294372368384)]} 0 1
[junit4:junit4] 2> 22105 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[139]} 0 3
[junit4:junit4] 2> 22109 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[140 (1428994294376562688)]} 0 1
[junit4:junit4] 2> 22112 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[140 (1428994294379708416)]} 0 1
[junit4:junit4] 2> 22116 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[141 (1428994294383902720)]} 0 1
[junit4:junit4] 2> 22121 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[141 (1428994294389145600)]} 0 1
[junit4:junit4] 2> 22121 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[141]} 0 3
[junit4:junit4] 2> 22125 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[142 (1428994294393339904)]} 0 1
[junit4:junit4] 2> 22130 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[142 (1428994294398582784)]} 0 1
[junit4:junit4] 2> 22130 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[142]} 0 3
[junit4:junit4] 2> 22134 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[143 (1428994294402777088)]} 0 1
[junit4:junit4] 2> 22139 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[143 (1428994294408019968)]} 0 1
[junit4:junit4] 2> 22139 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[143]} 0 3
[junit4:junit4] 2> 22143 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[144 (1428994294412214272)]} 0 1
[junit4:junit4] 2> 22148 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[144 (1428994294417457152)]} 0 1
[junit4:junit4] 2> 22148 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[144]} 0 3
[junit4:junit4] 2> 22152 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[145 (1428994294421651456)]} 0 1
[junit4:junit4] 2> 22155 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[145 (1428994294424797184)]} 0 1
[junit4:junit4] 2> 22159 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[146 (1428994294428991488)]} 0 1
[junit4:junit4] 2> 22165 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[146 (1428994294432137216)]} 0 1
[junit4:junit4] 2> 22165 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[146 (1428994294432137216)]} 0 4
[junit4:junit4] 2> 22187 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[147 (1428994294439477248)]} 0 19
[junit4:junit4] 2> 22194 T1964 C102 P12142 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[147 (1428994294465691648)]} 0 1
[junit4:junit4] 2> 22195 T1945 C99 P12128 /update {wt=javabin&version=2} {add=[147]} 0 4
[junit4:junit4] 2> 22198 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[148 (1428994294469885952)]} 0 1
[junit4:junit4] 2> 22207 T1977 C98 P12146 /update {distrib.from=http://127.0.0.1:12128/l_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[148 (1428994294476177408)]} 0 1
[junit4:junit4] 2> 22208 T1951 C99 P12128 /update {distrib.from=http://127.0.0.1:12142/l_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[148 (1428994294476177408)]} 0 5
[junit4:junit4] 2> 22208 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[148]} 0 7
[junit4:junit4] 2> 22212 T1925 C101 P12124 /update {wt=javabin&version=2} {add=[149 (1428994294484566016)]} 0 1
[junit4:junit4] 2> 22215 T1963 C102 P12142 /update {wt=javabin&version=2} {add=[149 (1428994294487711744)]} 0 1
[junit4:junit4] 2> 22215 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 22216 T1913 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22217 T1913 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 22220 T1925 C101 P12124 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22279 T1925 C101 P12124 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, segments_2, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1a9b10ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@61099381)),segFN=segments_3,generation=3,filenames=[_1_MockSep_0.frq, _1_Direct_0.pos, _1_Direct_0.doc, _0_MockSep_0.pos, _1.fnm, _1_Memory_0.ram, _0_MockSep_0.skp, _0_SimpleText_0.pst, _0.nvd, _1_Direct_0.tim, _0_MockSep_0.doc, _0.si, _1_Direct_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _1_MockSep_0.pos, _0_Disk_0.dvdd, _1_MockSep_0.tib, _0_Memory_0.ram, _1.nvd, _0.nvm, _0_Disk_0.dvdm, _1_MockSep_0.tii, _0_MockSep_0.tib, _1.fdx, _0_Direct_0.doc, _1_Disk_0.dvdd, _1.fdt, _1.si, _0_Direct_0.pos, _0_MockSep_0.frq, _0_Direct_0.tip, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_Direct_0.tim, _1_Disk_0.dvdm, segments_3, _0.fdt, _1_SimpleText_0.pst]
[junit4:junit4] 2> 22279 T1925 C101 P12124 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_1_MockSep_0.frq, _1_Direct_0.pos, _1_Direct_0.doc, _0_MockSep_0.pos, _1.fnm, _1_Memory_0.ram, _0_MockSep_0.skp, _0_SimpleText_0.pst, _0.nvd, _1_Direct_0.tim, _0_MockSep_0.doc, _0.si, _1_Direct_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _1_MockSep_0.pos, _0_Disk_0.dvdd, _1_MockSep_0.tib, _0_Memory_0.ram, _1.nvd, _0.nvm, _0_Disk_0.dvdm, _1_MockSep_0.tii, _0_MockSep_0.tib, _1.fdx, _0_Direct_0.doc, _1_Disk_0.dvdd, _1.fdt, _1.si, _0_Direct_0.pos, _0_MockSep_0.frq, _0_Direct_0.tip, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_Direct_0.tim, _1_Disk_0.dvdm, segments_3, _0.fdt, _1_SimpleText_0.pst]
[junit4:junit4] 2> 22287 T1925 C101 P12124 oass.SolrIndexSearcher.<init> Opening Searcher@38d364cd main
[junit4:junit4] 2> 22288 T1925 C101 P12124 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22288 T1939 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38d364cd main{StandardDirectoryReader(segments_3:5 _0(5.0):C1 _1(5.0):C66)}
[junit4:junit4] 2> 22289 T1925 C101 P12124 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 69
[junit4:junit4] 2> 22292 T1963 C102 P12142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22321 T1963 C102 P12142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1ff5ba65 lockFactory=org.apache.lucene.store.NativeFSLockFactory@67318fec)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 22321 T1963 C102 P12142 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 22327 T1963 C102 P12142 oass.SolrIndexSearcher.<init> Opening Searcher@612662bd main
[junit4:junit4] 2> 22327 T1963 C102 P12142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22328 T1973 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@612662bd main{StandardDirectoryReader(segments_3:3 _0(5.0):C35)}
[junit4:junit4] 2> 22328 T1963 C102 P12142 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:12128/l_/collection1/, StdNode: http://127.0.0.1:12146/l_/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 22331 T1979 C98 P12146 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22331 T1951 C99 P12128 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22357 T1951 C99 P12128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@219d8f1e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f407c07)),segFN=segments_4,generation=4,filenames=[_1_MockSep_0.frq, _1_Direct_0.pos, _1_Direct_0.doc, _0_MockSep_0.pos, _1.fnm, _1_Memory_0.ram, _0_MockSep_0.skp, _0_SimpleText_0.pst, _0.nvd, _1_Direct_0.tim, _0_MockSep_0.doc, _0.si, _1_Direct_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _1_MockSep_0.pos, _0_Disk_0.dvdd, _1_MockSep_0.tib, _0_Memory_0.ram, _1.nvd, _0.nvm, _0_Disk_0.dvdm, _1_MockSep_0.tii, _0_MockSep_0.tib, _1.fdx, _0_Direct_0.doc, _1_Disk_0.dvdd, _1.fdt, _1.si, _0_Direct_0.pos, _0_MockSep_0.frq, _0_Direct_0.tip, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_Direct_0.tim, _1_Disk_0.dvdm, _0.fdt, _1_SimpleText_0.pst, segments_4]
[junit4:junit4] 2> 22357 T1979 C98 P12146 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@9704ee lockFactory=org.apache.lucene.store.NativeFSLockFactory@44516d6d)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Disk_0.dvdd, _0_MockSep_0.pos, _0_Memory_0.ram, _0.nvm, _0_Disk_0.dvdm, _0_MockSep_0.tib, _0_Direct_0.doc, _0_MockSep_0.skp, _0.nvd, _0_SimpleText_0.pst, _0_MockSep_0.frq, _0_Direct_0.pos, _0.fdx, _0_Direct_0.tip, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0_Direct_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(or
[...truncated too long message...]
2795134735/jetty3/index.20130308181235305
[junit4:junit4] 2> 87322 T1987 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index.20130308181235305
[junit4:junit4] 2> 87322 T1987 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3
[junit4:junit4] 2> 87322 T1987 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3
[junit4:junit4] 2> 87322 T1987 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index
[junit4:junit4] 2> 87322 T1987 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362795134735/jetty3/index
[junit4:junit4] 2> 87323 T1987 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 87323 T1987 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/89312141950451722-127.0.0.1:12146_l__collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:153)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:243)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 87324 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 87324 T1987 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87324 T1987 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87324 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 87324 T1987 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87325 T1987 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 87325 T1987 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {foo_b=PostingsFormat(name=SimpleText), n_f1=PostingsFormat(name=Memory doPackFST= false), foo_d=PostingsFormat(name=Direct), foo_f=PostingsFormat(name=SimpleText), n_tl1=PostingsFormat(name=Memory doPackFST= false), n_d1=PostingsFormat(name=MockSep), rnd_b=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Memory doPackFST= false), n_td1=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=MockSep), id=PostingsFormat(name=Direct), range_facet_sl=PostingsFormat(name=Direct), range_facet_si=PostingsFormat(name=MockSep), oddField_s=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=MockSep), n_tf1=PostingsFormat(name=MockSep), n_dt1=PostingsFormat(name=Memory doPackFST= false), range_facet_l=PostingsFormat(name=MockSep), n_ti1=PostingsFormat(name=Direct), text=PostingsFormat(name=SimpleText), _version_=PostingsFormat(name=Memory doPackFST= false), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=MockSep), n_tdt1=PostingsFormat(name=Memory doPackFST= false), other_tl1=PostingsFormat(name=Memory doPackFST= false), n_l1=PostingsFormat(name=MockSep), a_si=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=ar_MA, timezone=America/Dawson
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=162495520,total=451936256
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestMergePolicyConfig, NoCacheHeaderTest, TestZkChroot, DirectUpdateHandlerTest, TimeZoneUtilsTest, BadIndexSchemaTest, TestNumberUtils, TestSolrDiscoveryProperties, SolrCmdDistributorTest, HighlighterConfigTest, TermVectorComponentDistributedTest, TestFiltering, FileUtilsTest, TestBinaryField, TestQuerySenderNoQuery, SimplePostToolTest, TestIndexingPerformance, TestLMJelinekMercerSimilarityFactory, TestSolrXMLSerializer, XmlUpdateRequestHandlerTest, TestRangeQuery, TestDFRSimilarityFactory, TestAnalyzedSuggestions, DocumentAnalysisRequestHandlerTest, TestCollationField, IndexBasedSpellCheckerTest, TestJmxIntegration, SpellCheckComponentTest, TestLazyCores, TestBinaryResponseWriter, CollectionsAPIDistributedZkTest, TestUpdate, TestRTGBase, BasicZkTest, ChaosMonkeySafeLeaderTest, DocValuesMultiTest, StatsComponentTest, SuggesterWFSTTest, TestSystemIdResolver, TestSuggestSpellingConverter, DirectSolrSpellCheckerTest, URLClassifyProcessorTest, TestHashPartitioner, TestBM25SimilarityFactory, TestJmxMonitoredMap, HardAutoCommitTest, TestStressReorder, TestFoldingMultitermQuery, TestDistributedGrouping, SolrIndexConfigTest, SampleTest, DisMaxRequestHandlerTest, JSONWriterTest, DOMUtilTest, TestDefaultSimilarityFactory, ConvertedLegacyTest, AliasIntegrationTest, LeaderElectionIntegrationTest, TestFastOutputStream, SolrCoreCheckLockOnStartupTest, PathHierarchyTokenizerFactoryTest, TestComponentsName, EchoParamsTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 88.20s, 1 test, 1 error <<< FAILURES!
[...truncated 483 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 269 suites, 1137 tests, 1 error, 27 ignored (8 assumptions)
Total time: 51 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3800 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3800/
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1659, name=coreLoadExecutor-710-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1659, name=coreLoadExecutor-710-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
at __randomizedtesting.SeedInfo.seed([97FCCAC348BCDC9B]:0)
at java.lang.Thread.getStackTrace(Thread.java:1567)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:980)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:878)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:870)
at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:34)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:744)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:874)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:580)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:575)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.AssertionError: Directory not closed: BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@674f59da lockFactory=org.apache.lucene.store.NativeFSLockFactory@5e1cdb2)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
... 1 more
Build Log:
[...truncated 9050 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 0 T1576 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 5 T1576 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1362994925112
[junit4:junit4] 2> 6 T1576 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 7 T1577 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 10 T1577 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 10 T1577 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 10 T1577 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 11 T1577 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 12 T1577 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-1362994925111/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 108 T1576 oasc.ZkTestServer.run start zk server on port:31684
[junit4:junit4] 2> 108 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@182f9224
[junit4:junit4] 2> 109 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 110 T1582 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 110 T1582 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 111 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31685
[junit4:junit4] 2> 111 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31685
[junit4:junit4] 2> 111 T1580 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 114 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0000 with negotiated timeout 10000 for client /140.211.11.196:31685
[junit4:junit4] 2> 114 T1582 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0000, negotiated timeout = 10000
[junit4:junit4] 2> 114 T1583 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@182f9224 name:ZooKeeperConnection Watcher:127.0.0.1:31684 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 115 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 115 T1576 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 149 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0000
[junit4:junit4] 2> 161 T1583 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 162 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31685 which had sessionid 0x13d58d40e3f0000
[junit4:junit4] 2> 161 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0000 closed
[junit4:junit4] 2> 162 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e822356
[junit4:junit4] 2> 163 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 164 T1584 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 164 T1584 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 164 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31686
[junit4:junit4] 2> 165 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31686
[junit4:junit4] 2> 166 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0001 with negotiated timeout 10000 for client /140.211.11.196:31686
[junit4:junit4] 2> 166 T1584 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0001, negotiated timeout = 10000
[junit4:junit4] 2> 166 T1585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e822356 name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 167 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 167 T1576 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 182 T1576 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 184 T1576 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 214 T1576 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 216 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 217 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 228 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 229 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 335 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 336 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 338 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 339 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 350 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 350 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 353 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 354 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 357 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 358 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 360 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 361 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 364 T1576 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 364 T1576 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 371 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0001
[junit4:junit4] 2> 373 T1585 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 373 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31686 which had sessionid 0x13d58d40e3f0001
[junit4:junit4] 2> 373 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0001 closed
[junit4:junit4] 2> 562 T1576 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 568 T1576 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:31687
[junit4:junit4] 2> 569 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 569 T1576 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 570 T1576 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479
[junit4:junit4] 2> 570 T1576 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/solr.xml
[junit4:junit4] 2> 570 T1576 oasc.CoreContainer.<init> New CoreContainer 1585737606
[junit4:junit4] 2> 571 T1576 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/'
[junit4:junit4] 2> 572 T1576 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/'
[junit4:junit4] 2> 635 T1576 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 635 T1576 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 636 T1576 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 636 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 637 T1576 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 637 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 638 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 638 T1576 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 639 T1576 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 639 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 661 T1576 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 681 T1576 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:31684/solr
[junit4:junit4] 2> 681 T1576 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 682 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@57a52ec9
[junit4:junit4] 2> 683 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 683 T1595 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 684 T1595 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 684 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31688
[junit4:junit4] 2> 685 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31688
[junit4:junit4] 2> 686 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0002 with negotiated timeout 20000 for client /140.211.11.196:31688
[junit4:junit4] 2> 686 T1595 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0002, negotiated timeout = 20000
[junit4:junit4] 2> 687 T1596 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57a52ec9 name:ZooKeeperConnection Watcher:127.0.0.1:31684 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 687 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 688 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0002
[junit4:junit4] 2> 703 T1596 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 703 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31688 which had sessionid 0x13d58d40e3f0002
[junit4:junit4] 2> 703 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0002 closed
[junit4:junit4] 2> 704 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 716 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3dbdd421
[junit4:junit4] 2> 717 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 717 T1597 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 718 T1597 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 718 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31689
[junit4:junit4] 2> 718 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31689
[junit4:junit4] 2> 719 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0003 with negotiated timeout 20000 for client /140.211.11.196:31689
[junit4:junit4] 2> 719 T1597 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0003, negotiated timeout = 20000
[junit4:junit4] 2> 720 T1598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dbdd421 name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 720 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 722 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 730 T1576 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 732 T1576 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31687_
[junit4:junit4] 2> 733 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:31687_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:31687_
[junit4:junit4] 2> 734 T1576 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31687_
[junit4:junit4] 2> 742 T1576 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 751 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 759 T1576 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 762 T1576 oasc.Overseer.start Overseer (id=89325235412598787-127.0.0.1:31687_-n_0000000000) starting
[junit4:junit4] 2> 762 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 764 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 765 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 775 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 777 T1600 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 777 T1576 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 788 T1576 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 790 T1576 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 792 T1599 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 796 T1601 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/collection1
[junit4:junit4] 2> 796 T1601 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 797 T1601 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 797 T1601 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 798 T1601 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/collection1/'
[junit4:junit4] 2> 799 T1601 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/collection1/lib/README' to classloader
[junit4:junit4] 2> 800 T1601 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 849 T1601 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 908 T1601 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1009 T1601 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 1016 T1601 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1127 T3 oasu.ConcurrentLRUCache.finalize SEVERE ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
[junit4:junit4] 2> 1677 T1601 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1688 T1601 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1693 T1601 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1705 T1601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1710 T1601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1714 T1601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1715 T1601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1716 T1601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1716 T1601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1717 T1601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1717 T1601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1718 T1601 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1718 T1601 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1362994925479/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data/
[junit4:junit4] 2> 1718 T1601 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4437601e
[junit4:junit4] 2> 1719 T1601 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1719 T1601 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data forceNew: false
[junit4:junit4] 2> 1720 T1601 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data/index/
[junit4:junit4] 2> 1720 T1601 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1720 T1601 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data/index forceNew: false
[junit4:junit4] 2> 1724 T1601 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ab54d6e lockFactory=org.apache.lucene.store.NativeFSLockFactory@43956647),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1724 T1601 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1727 T1601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1727 T1601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1728 T1601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1728 T1601 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1729 T1601 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1729 T1601 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1729 T1601 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1730 T1601 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1730 T1601 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1743 T1601 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1749 T1601 oass.SolrIndexSearcher.<init> Opening Searcher@46e5d23a main
[junit4:junit4] 2> 1750 T1601 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/control/data/tlog
[junit4:junit4] 2> 1750 T1601 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1751 T1601 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1754 T1602 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46e5d23a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1756 T1601 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1756 T1601 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1757 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 2295 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2296 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:31687_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:31687"}
[junit4:junit4] 2> 2296 T1599 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2296 T1599 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 2297 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 2311 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2871 T1601 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2871 T1601 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:31687 collection:control_collection shard:shard1
[junit4:junit4] 2> 2872 T1601 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2890 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2897 T1601 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2897 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2899 T1601 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2900 T1601 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2900 T1601 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:31687/collection1/
[junit4:junit4] 2> 2900 T1601 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2900 T1601 oasc.SyncStrategy.syncToMe http://127.0.0.1:31687/collection1/ has no replicas
[junit4:junit4] 2> 2901 T1601 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:31687/collection1/
[junit4:junit4] 2> 2901 T1601 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2905 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3820 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3838 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3886 T1601 oasc.ZkController.register We are http://127.0.0.1:31687/collection1/ and leader is http://127.0.0.1:31687/collection1/
[junit4:junit4] 2> 3886 T1601 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:31687
[junit4:junit4] 2> 3886 T1601 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3886 T1601 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3887 T1601 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3888 T1601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3889 T1576 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3890 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3891 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3907 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3908 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@c72eb11
[junit4:junit4] 2> 3909 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3909 T1604 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3910 T1604 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 3910 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31690
[junit4:junit4] 2> 3910 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31690
[junit4:junit4] 2> 3911 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0004 with negotiated timeout 10000 for client /140.211.11.196:31690
[junit4:junit4] 2> 3911 T1604 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0004, negotiated timeout = 10000
[junit4:junit4] 2> 3912 T1605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c72eb11 name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3912 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3913 T1576 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3915 T1576 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4108 T1576 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4111 T1576 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:31693
[junit4:junit4] 2> 4112 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4112 T1576 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4113 T1576 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021
[junit4:junit4] 2> 4113 T1576 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/solr.xml
[junit4:junit4] 2> 4114 T1576 oasc.CoreContainer.<init> New CoreContainer 2050083328
[junit4:junit4] 2> 4115 T1576 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/'
[junit4:junit4] 2> 4115 T1576 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/'
[junit4:junit4] 2> 4177 T1576 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4178 T1576 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4178 T1576 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4179 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4179 T1576 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4180 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4180 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4181 T1576 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4181 T1576 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4182 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4203 T1576 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4223 T1576 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:31684/solr
[junit4:junit4] 2> 4223 T1576 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4224 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d58a9d9
[junit4:junit4] 2> 4225 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4225 T1615 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4226 T1615 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 4227 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31694
[junit4:junit4] 2> 4227 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31694
[junit4:junit4] 2> 4229 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0005 with negotiated timeout 20000 for client /140.211.11.196:31694
[junit4:junit4] 2> 4229 T1615 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0005, negotiated timeout = 20000
[junit4:junit4] 2> 4229 T1616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d58a9d9 name:ZooKeeperConnection Watcher:127.0.0.1:31684 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4229 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4230 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0005
[junit4:junit4] 2> 4231 T1616 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4231 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31694 which had sessionid 0x13d58d40e3f0005
[junit4:junit4] 2> 4231 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0005 closed
[junit4:junit4] 2> 4232 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4244 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@50cd494f
[junit4:junit4] 2> 4245 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4245 T1617 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4246 T1617 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 4246 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31695
[junit4:junit4] 2> 4247 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31695
[junit4:junit4] 2> 4254 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0006 with negotiated timeout 20000 for client /140.211.11.196:31695
[junit4:junit4] 2> 4254 T1617 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0006, negotiated timeout = 20000
[junit4:junit4] 2> 4254 T1618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50cd494f name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4255 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4255 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4257 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4259 T1576 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5261 T1576 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31693_
[junit4:junit4] 2> 5262 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:31693_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:31693_
[junit4:junit4] 2> 5265 T1576 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31693_
[junit4:junit4] 2> 5268 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 5269 T1605 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5269 T1618 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5270 T1598 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5288 T1619 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/collection1
[junit4:junit4] 2> 5289 T1619 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 5289 T1619 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 5290 T1619 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 5291 T1619 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/collection1/'
[junit4:junit4] 2> 5292 T1619 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/collection1/lib/README' to classloader
[junit4:junit4] 2> 5293 T1619 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 5342 T1619 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 5343 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 5343 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:31687_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:31687"}
[junit4:junit4] 2> 5360 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5360 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5360 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5404 T1619 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5504 T1619 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5512 T1619 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6137 T1619 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6149 T1619 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6153 T1619 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6164 T1619 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6169 T1619 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6174 T1619 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6175 T1619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6175 T1619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6175 T1619 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6176 T1619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6176 T1619 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6176 T1619 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6177 T1619 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1362994929021/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1/
[junit4:junit4] 2> 6177 T1619 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4437601e
[junit4:junit4] 2> 6178 T1619 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6178 T1619 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1 forceNew: false
[junit4:junit4] 2> 6178 T1619 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1/index/
[junit4:junit4] 2> 6179 T1619 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6179 T1619 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1/index forceNew: false
[junit4:junit4] 2> 6182 T1619 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b24b783 lockFactory=org.apache.lucene.store.NativeFSLockFactory@681dadc7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6182 T1619 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6185 T1619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6185 T1619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6186 T1619 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6187 T1619 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6187 T1619 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6188 T1619 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6188 T1619 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6189 T1619 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6189 T1619 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6202 T1619 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6207 T1619 oass.SolrIndexSearcher.<init> Opening Searcher@4b948582 main
[junit4:junit4] 2> 6208 T1619 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty1/tlog
[junit4:junit4] 2> 6209 T1619 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6209 T1619 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6213 T1620 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b948582 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6215 T1619 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 6215 T1619 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6873 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6874 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:31693_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:31693"}
[junit4:junit4] 2> 6874 T1599 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 6874 T1599 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6889 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6889 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6889 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7244 T1619 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7244 T1619 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:31693 collection:collection1 shard:shard1
[junit4:junit4] 2> 7245 T1619 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7261 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7262 T1619 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7263 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7265 T1619 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7265 T1619 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7266 T1619 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:31693/collection1/
[junit4:junit4] 2> 7266 T1619 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7266 T1619 oasc.SyncStrategy.syncToMe http://127.0.0.1:31693/collection1/ has no replicas
[junit4:junit4] 2> 7266 T1619 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:31693/collection1/
[junit4:junit4] 2> 7267 T1619 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7270 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8393 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8419 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8419 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8419 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8446 T1619 oasc.ZkController.register We are http://127.0.0.1:31693/collection1/ and leader is http://127.0.0.1:31693/collection1/
[junit4:junit4] 2> 8446 T1619 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:31693
[junit4:junit4] 2> 8446 T1619 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8446 T1619 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8447 T1619 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8448 T1619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8449 T1576 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 8450 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8450 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8658 T1576 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8661 T1576 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:48298
[junit4:junit4] 2> 8662 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8663 T1576 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8663 T1576 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572
[junit4:junit4] 2> 8664 T1576 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/solr.xml
[junit4:junit4] 2> 8664 T1576 oasc.CoreContainer.<init> New CoreContainer 604822622
[junit4:junit4] 2> 8665 T1576 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/'
[junit4:junit4] 2> 8665 T1576 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/'
[junit4:junit4] 2> 8727 T1576 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8728 T1576 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8729 T1576 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8729 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8729 T1576 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8730 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8730 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8731 T1576 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8731 T1576 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8732 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8753 T1576 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8773 T1576 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:31684/solr
[junit4:junit4] 2> 8773 T1576 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8774 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c423646
[junit4:junit4] 2> 8775 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8775 T1631 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8776 T1631 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 8776 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:52414
[junit4:junit4] 2> 8777 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:52414
[junit4:junit4] 2> 8779 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0007 with negotiated timeout 20000 for client /140.211.11.196:52414
[junit4:junit4] 2> 8779 T1631 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0007, negotiated timeout = 20000
[junit4:junit4] 2> 8779 T1632 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c423646 name:ZooKeeperConnection Watcher:127.0.0.1:31684 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8779 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8780 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0007
[junit4:junit4] 2> 8788 T1632 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8788 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:52414 which had sessionid 0x13d58d40e3f0007
[junit4:junit4] 2> 8788 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0007 closed
[junit4:junit4] 2> 8789 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8800 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3886f27b
[junit4:junit4] 2> 8801 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8802 T1633 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8802 T1633 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 8802 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:60224
[junit4:junit4] 2> 8803 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:60224
[junit4:junit4] 2> 8804 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0008 with negotiated timeout 20000 for client /140.211.11.196:60224
[junit4:junit4] 2> 8804 T1633 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0008, negotiated timeout = 20000
[junit4:junit4] 2> 8804 T1634 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3886f27b name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8804 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8805 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8811 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8813 T1576 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9816 T1576 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48298_
[junit4:junit4] 2> 9818 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f0008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48298_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48298_
[junit4:junit4] 2> 9819 T1576 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48298_
[junit4:junit4] 2> 9843 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9844 T1618 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9844 T1605 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9844 T1634 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9844 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9844 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9845 T1598 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9851 T1635 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/collection1
[junit4:junit4] 2> 9852 T1635 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9852 T1635 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9852 T1635 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9854 T1635 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/collection1/'
[junit4:junit4] 2> 9855 T1635 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9855 T1635 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/collection1/lib/README' to classloader
[junit4:junit4] 2> 9906 T1635 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 9942 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9943 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:31693_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:31693"}
[junit4:junit4] 2> 9967 T1635 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 10058 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10058 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10058 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10094 T1634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10094 T1635 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 10102 T1635 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 10736 T1635 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 10747 T1635 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 10751 T1635 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 10764 T1635 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10769 T1635 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10773 T1635 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10774 T1635 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10775 T1635 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10775 T1635 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10776 T1635 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10776 T1635 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10777 T1635 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10777 T1635 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/
[junit4:junit4] 2> 10777 T1635 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4437601e
[junit4:junit4] 2> 10778 T1635 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10778 T1635 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2 forceNew: false
[junit4:junit4] 2> 10779 T1635 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index/
[junit4:junit4] 2> 10779 T1635 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 10779 T1635 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index forceNew: false
[junit4:junit4] 2> 10782 T1635 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2260964a lockFactory=org.apache.lucene.store.NativeFSLockFactory@55bdbc3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 10783 T1635 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 10785 T1635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 10786 T1635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 10786 T1635 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 10787 T1635 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 10788 T1635 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 10788 T1635 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 10788 T1635 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 10789 T1635 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 10789 T1635 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 10802 T1635 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 10808 T1635 oass.SolrIndexSearcher.<init> Opening Searcher@5a7fe256 main
[junit4:junit4] 2> 10808 T1635 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/tlog
[junit4:junit4] 2> 10809 T1635 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 10810 T1635 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 10814 T1636 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a7fe256 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 10816 T1635 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 10816 T1635 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11608 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11609 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:48298_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48298"}
[junit4:junit4] 2> 11609 T1599 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 11609 T1599 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 11656 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11656 T1634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11656 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11656 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11837 T1635 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11837 T1635 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48298 collection:collection1 shard:shard1
[junit4:junit4] 2> 11865 T1635 oasc.ZkController.register We are http://127.0.0.1:48298/collection1/ and leader is http://127.0.0.1:31693/collection1/
[junit4:junit4] 2> 11865 T1635 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48298
[junit4:junit4] 2> 11865 T1635 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 11866 T1635 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C73 name=collection1 org.apache.solr.core.SolrCore@65c3b1c5 url=http://127.0.0.1:48298/collection1 node=127.0.0.1:48298_ C73_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48298_, base_url=http://127.0.0.1:48298}
[junit4:junit4] 2> 11866 T1637 C73 P48298 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 11866 T1635 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11867 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 11867 T1637 C73 P48298 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 11867 T1637 C73 P48298 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11868 T1576 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 11869 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 11869 T1637 C73 P48298 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11869 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11887 T1576 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C74 name=collection1 org.apache.solr.core.SolrCore@5d00f1ac url=http://127.0.0.1:31687/collection1 node=127.0.0.1:31687_ C74_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:31687_, base_url=http://127.0.0.1:31687, leader=true}
[junit4:junit4] 2> 11901 T1588 C74 P31687 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ab54d6e lockFactory=org.apache.lucene.store.NativeFSLockFactory@43956647),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 11901 T1588 C74 P31687 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 11905 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[1 (1429203779064954880)]} 0 7
[junit4:junit4] 2> 11905 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10001 (1429203779064954881)]} 0 7
[junit4:junit4] 2> ASYNC NEW_CORE C75 name=collection1 org.apache.solr.core.SolrCore@591d455a url=http://127.0.0.1:31693/collection1 node=127.0.0.1:31693_ C75_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:31693_, base_url=http://127.0.0.1:31693, leader=true}
[junit4:junit4] 2> 11927 T1610 C75 P31693 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b24b783 lockFactory=org.apache.lucene.store.NativeFSLockFactory@681dadc7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 11928 T1610 C75 P31693 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 11930 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[1 (1429203779092217857)]} 0 6
[junit4:junit4] 2> 11930 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10001 (1429203779092217856)]} 0 6
[junit4:junit4] 2> 12011 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10002 (1429203779181346816)]} 0 1
[junit4:junit4] 2> 12011 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[2 (1429203779182395392)]} 0 1
[junit4:junit4] 2> 12015 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[2 (1429203779185541120)]} 0 1
[junit4:junit4] 2> 12015 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10002 (1429203779185541121)]} 0 1
[junit4:junit4] 2> 12081 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[1 (-1429203779255795712)]} 0 0
[junit4:junit4] 2> 12083 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10001 (-1429203779256844288)]} 0 1
[junit4:junit4] 2> 12084 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[1 (-1429203779258941440)]} 0 0
[junit4:junit4] 2> 12086 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10001 (-1429203779259990016)]} 0 1
[junit4:junit4] 2> 12088 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[3 (1429203779262087168)]} 0 1
[junit4:junit4] 2> 12090 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10003 (1429203779264184320)]} 0 1
[junit4:junit4] 2> 12091 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[3 (1429203779265232896)]} 0 1
[junit4:junit4] 2> 12094 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10003 (1429203779268378624)]} 0 1
[junit4:junit4] 2> 12165 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[2 (-1429203779343876096)]} 0 0
[junit4:junit4] 2> 12169 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10002 (-1429203779347021824)]} 0 1
[junit4:junit4] 2> 12169 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[2 (-1429203779347021824)]} 0 1
[junit4:junit4] 2> 12171 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10002 (-1429203779350167552)]} 0 0
[junit4:junit4] 2> 12172 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[4 (1429203779350167552)]} 0 1
[junit4:junit4] 2> 12175 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[4 (1429203779353313280)]} 0 1
[junit4:junit4] 2> 12175 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10004 (1429203779353313280)]} 0 1
[junit4:junit4] 2> 12179 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10004 (1429203779357507584)]} 0 1
[junit4:junit4] 2> 12215 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[4 (-1429203779396304896)]} 0 0
[junit4:junit4] 2> 12218 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[4 (-1429203779399450624)]} 0 0
[junit4:junit4] 2> 12219 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10004 (-1429203779400499200)]} 0 0
[junit4:junit4] 2> 12222 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[5 (1429203779402596352)]} 0 1
[junit4:junit4] 2> 12222 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10004 (-1429203779403644928)]} 0 0
[junit4:junit4] 2> 12226 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10005 (1429203779406790656)]} 0 1
[junit4:junit4] 2> 12226 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[5 (1429203779406790656)]} 0 1
[junit4:junit4] 2> 12229 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10005 (1429203779409936384)]} 0 1
[junit4:junit4] 2> 12258 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[5 (-1429203779441393664)]} 0 0
[junit4:junit4] 2> 12261 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10005 (-1429203779444539392)]} 0 0
[junit4:junit4] 2> 12262 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[5 (-1429203779444539392)]} 0 1
[junit4:junit4] 2> 12264 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10005 (-1429203779447685120)]} 0 0
[junit4:junit4] 2> 12265 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[6 (1429203779447685120)]} 0 1
[junit4:junit4] 2> 12268 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[6 (1429203779450830848)]} 0 1
[junit4:junit4] 2> 12268 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10006 (1429203779450830848)]} 0 1
[junit4:junit4] 2> 12272 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10006 (1429203779455025152)]} 0 1
[junit4:junit4] 2> 12327 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[7 (1429203779512696832)]} 0 1
[junit4:junit4] 2> 12331 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10007 (1429203779516891136)]} 0 1
[junit4:junit4] 2> 12331 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[7 (1429203779516891136)]} 0 1
[junit4:junit4] 2> 12335 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10007 (1429203779521085440)]} 0 1
[junit4:junit4] 2> 12402 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[8 (1429203779591340032)]} 0 1
[junit4:junit4] 2> 12405 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10008 (1429203779594485760)]} 0 1
[junit4:junit4] 2> 12405 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[8 (1429203779594485760)]} 0 1
[junit4:junit4] 2> 12408 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10008 (1429203779597631488)]} 0 1
[junit4:junit4] 2> 12440 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[6 (-1429203779632234496)]} 0 0
[junit4:junit4] 2> 12442 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10006 (-1429203779634331648)]} 0 0
[junit4:junit4] 2> 12443 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[6 (-1429203779635380224)]} 0 0
[junit4:junit4] 2> 12445 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10006 (-1429203779637477376)]} 0 0
[junit4:junit4] 2> 12447 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[9 (1429203779638525952)]} 0 1
[junit4:junit4] 2> 12450 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[9 (1429203779641671680)]} 0 1
[junit4:junit4] 2> 12451 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10009 (1429203779642720256)]} 0 1
[junit4:junit4] 2> 12454 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10009 (1429203779645865984)]} 0 1
[junit4:junit4] 2> 12550 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10 (1429203779746529280)]} 0 1
[junit4:junit4] 2> 12553 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10 (1429203779749675008)]} 0 1
[junit4:junit4] 2> 12553 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10010 (1429203779749675008)]} 0 1
[junit4:junit4] 2> 12557 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10010 (1429203779753869312)]} 0 1
[junit4:junit4] 2> 12573 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[11 (1429203779770646528)]} 0 1
[junit4:junit4] 2> 12576 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[11 (1429203779773792256)]} 0 1
[junit4:junit4] 2> 12577 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10011 (1429203779774840832)]} 0 1
[junit4:junit4] 2> 12580 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10011 (1429203779777986560)]} 0 1
[junit4:junit4] 2> 12646 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[12 (1429203779847192576)]} 0 1
[junit4:junit4] 2> 12649 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[12 (1429203779850338304)]} 0 1
[junit4:junit4] 2> 12650 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10012 (1429203779851386880)]} 0 1
[junit4:junit4] 2> 12653 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10012 (1429203779854532608)]} 0 1
[junit4:junit4] 2> 12655 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[7 (-1429203779857678336)]} 0 0
[junit4:junit4] 2> 12658 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[7 (-1429203779860824064)]} 0 0
[junit4:junit4] 2> 12659 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10007 (-1429203779861872640)]} 0 0
[junit4:junit4] 2> 12662 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[13 (1429203779863969792)]} 0 1
[junit4:junit4] 2> 12662 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10007 (-1429203779865018368)]} 0 0
[junit4:junit4] 2> 12665 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[13 (1429203779867115520)]} 0 1
[junit4:junit4] 2> 12666 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10013 (1429203779868164096)]} 0 1
[junit4:junit4] 2> 12669 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10013 (1429203779871309824)]} 0 1
[junit4:junit4] 2> 12698 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[14 (1429203779901718528)]} 0 1
[junit4:junit4] 2> 12701 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10014 (1429203779904864256)]} 0 1
[junit4:junit4] 2> 12701 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[14 (1429203779904864256)]} 0 1
[junit4:junit4] 2> 12705 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10014 (1429203779909058560)]} 0 1
[junit4:junit4] 2> 12737 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[15 (1429203779942612992)]} 0 1
[junit4:junit4] 2> 12740 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[15 (1429203779945758720)]} 0 1
[junit4:junit4] 2> 12741 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10015 (1429203779946807296)]} 0 1
[junit4:junit4] 2> 12744 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10015 (1429203779949953024)]} 0 1
[junit4:junit4] 2> 12835 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[9 (-1429203780045373440)]} 0 1
[junit4:junit4] 2> 12837 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[9 (-1429203780048519168)]} 0 0
[junit4:junit4] 2> 12838 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10009 (-1429203780049567744)]} 0 0
[junit4:junit4] 2> 12841 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[16 (1429203780051664896)]} 0 1
[junit4:junit4] 2> 12842 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10009 (-1429203780052713472)]} 0 1
[junit4:junit4] 2> 12846 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10016 (1429203780056907776)]} 0 1
[junit4:junit4] 2> 12846 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[16 (1429203780056907776)]} 0 1
[junit4:junit4] 2> 12849 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10016 (1429203780060053504)]} 0 1
[junit4:junit4] 2> 12929 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[17 (1429203780143939584)]} 0 1
[junit4:junit4] 2> 12931 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10017 (1429203780146036736)]} 0 1
[junit4:junit4] 2> 12932 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[17 (1429203780147085312)]} 0 1
[junit4:junit4] 2> 12934 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10017 (1429203780149182464)]} 0 1
[junit4:junit4] 2> 12978 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[18 (1429203780195319808)]} 0 1
[junit4:junit4] 2> 12981 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10018 (1429203780198465536)]} 0 1
[junit4:junit4] 2> 12981 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[18 (1429203780198465536)]} 0 1
[junit4:junit4] 2> 12985 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10018 (1429203780202659840)]} 0 1
[junit4:junit4] 2> 13042 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10 (-1429203780263477248)]} 0 0
[junit4:junit4] 2> 13045 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10 (-1429203780266622976)]} 0 0
[junit4:junit4] 2> 13046 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10010 (-1429203780267671552)]} 0 0
[junit4:junit4] 2> 13049 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[19 (1429203780269768704)]} 0 1
[junit4:junit4] 2> 13049 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10010 (-1429203780270817280)]} 0 0
[junit4:junit4] 2> 13053 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10019 (1429203780273963008)]} 0 1
[junit4:junit4] 2> 13053 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[19 (1429203780273963008)]} 0 1
[junit4:junit4] 2> 13056 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10019 (1429203780277108736)]} 0 1
[junit4:junit4] 2> 13086 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[20 (1429203780308566016)]} 0 1
[junit4:junit4] 2> 13089 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10020 (1429203780311711744)]} 0 1
[junit4:junit4] 2> 13089 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[20 (1429203780311711744)]} 0 1
[junit4:junit4] 2> 13093 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10020 (1429203780315906048)]} 0 1
[junit4:junit4] 2> 13159 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13160 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:48298_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48298"}
[junit4:junit4] 2> 13171 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[13 (-1429203780398743552)]} 0 0
[junit4:junit4] 2> 13174 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[13 (-1429203780401889280)]} 0 0
[junit4:junit4] 2> 13175 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10013 (-1429203780402937856)]} 0 0
[junit4:junit4] 2> 13178 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10013 (-1429203780406083584)]} 0 0
[junit4:junit4] 2> 13179 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[21 (1429203780406083584)]} 0 1
[junit4:junit4] 2> 13181 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13181 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13181 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13181 T1634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13183 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10021 (1429203780410277888)]} 0 1
[junit4:junit4] 2> 13183 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[21 (1429203780410277888)]} 0 1
[junit4:junit4] 2> C73_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48298_, base_url=http://127.0.0.1:48298}
[junit4:junit4] 2> 13197 T1624 C73 P48298 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2260964a lockFactory=org.apache.lucene.store.NativeFSLockFactory@55bdbc3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 13198 T1624 C73 P48298 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 13200 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10021 (1429203780414472192)]} 0 5
[junit4:junit4] 2> 13200 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10021 (1429203780414472192)]} 0 14
[junit4:junit4] 2> 13267 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[22 (1429203780498358272)]} 0 1
[junit4:junit4] 2> 13274 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1429203780502552576)]} 0 1
[junit4:junit4] 2> 13275 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[22 (1429203780502552576)]} 0 5
[junit4:junit4] 2> 13279 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[16 (-1429203780511989760)]} 0 0
[junit4:junit4] 2> 13284 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10022 (1429203780516184064)]} 0 1
[junit4:junit4] 2> 13285 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[16 (-1429203780515135488)]} 0 0
[junit4:junit4] 2> 13286 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[16 (-1429203780515135488)]} 0 4
[junit4:junit4] 2> 13290 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[23 (1429203780522475520)]} 0 1
[junit4:junit4] 2> 13290 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10022 (1429203780519329792)]} 0 1
[junit4:junit4] 2> 13291 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10022 (1429203780519329792)]} 0 5
[junit4:junit4] 2> 13295 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10016 (-1429203780528766976)]} 0 0
[junit4:junit4] 2> 13296 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1429203780525621248)]} 0 1
[junit4:junit4] 2> 13297 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[23 (1429203780525621248)]} 0 5
[junit4:junit4] 2> 13300 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10016 (-1429203780531912704)]} 0 0
[junit4:junit4] 2> 13301 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10016 (-1429203780531912704)]} 0 3
[junit4:junit4] 2> 13305 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10023 (1429203780538204160)]} 0 1
[junit4:junit4] 2> 13312 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10023 (1429203780542398464)]} 0 1
[junit4:junit4] 2> 13312 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10023 (1429203780542398464)]} 0 4
[junit4:junit4] 2> 13357 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[24 (1429203780592730112)]} 0 1
[junit4:junit4] 2> 13364 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1429203780596924416)]} 0 1
[junit4:junit4] 2> 13364 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[24 (1429203780596924416)]} 0 4
[junit4:junit4] 2> 13373 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10024 (1429203780609507328)]} 0 1
[junit4:junit4] 2> 13379 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10024 (1429203780612653056)]} 0 1
[junit4:junit4] 2> 13379 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10024 (1429203780612653056)]} 0 4
[junit4:junit4] 2> 13427 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[25 (1429203780666130432)]} 0 1
[junit4:junit4] 2> 13434 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1429203780670324736)]} 0 1
[junit4:junit4] 2> 13434 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[25 (1429203780670324736)]} 0 4
[junit4:junit4] 2> 13442 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10025 (1429203780681859072)]} 0 1
[junit4:junit4] 2> 13449 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10025 (1429203780686053376)]} 0 1
[junit4:junit4] 2> 13449 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10025 (1429203780686053376)]} 0 4
[junit4:junit4] 2> 13496 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[26 (1429203780738482176)]} 0 1
[junit4:junit4] 2> 13502 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1429203780741627904)]} 0 1
[junit4:junit4] 2> 13502 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[26 (1429203780741627904)]} 0 4
[junit4:junit4] 2> 13511 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10026 (1429203780754210816)]} 0 1
[junit4:junit4] 2> 13517 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10026 (1429203780757356544)]} 0 1
[junit4:junit4] 2> 13517 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10026 (1429203780757356544)]} 0 4
[junit4:junit4] 2> 13604 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[27 (1429203780851728384)]} 0 1
[junit4:junit4] 2> 13611 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1429203780855922688)]} 0 1
[junit4:junit4] 2> 13611 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[27 (1429203780855922688)]} 0 4
[junit4:junit4] 2> 13619 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10027 (1429203780867457024)]} 0 1
[junit4:junit4] 2> 13626 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10027 (1429203780871651328)]} 0 1
[junit4:junit4] 2> 13626 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10027 (1429203780871651328)]} 0 4
[junit4:junit4] 2> 13629 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[28 (1429203780877942784)]} 0 1
[junit4:junit4] 2> 13636 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1429203780882137088)]} 0 1
[junit4:junit4] 2> 13636 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[28 (1429203780882137088)]} 0 4
[junit4:junit4] 2> 13644 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10028 (1429203780893671424)]} 0 1
[junit4:junit4] 2> 13651 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10028 (1429203780897865728)]} 0 1
[junit4:junit4] 2> 13651 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10028 (1429203780897865728)]} 0 4
[junit4:junit4] 2> 13734 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[19 (-1429203780989091840)]} 0 0
[junit4:junit4] 2> 13740 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[19 (-1429203780992237568)]} 0 0
[junit4:junit4] 2> 13741 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[19 (-1429203780992237568)]} 0 4
[junit4:junit4] 2> 13745 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[29 (1429203780999577600)]} 0 1
[junit4:junit4] 2> 13749 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10019 (-1429203781004820480)]} 0 0
[junit4:junit4] 2> 13752 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1429203781003771904)]} 0 1
[junit4:junit4] 2> 13752 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[29 (1429203781003771904)]} 0 4
[junit4:junit4] 2> 13756 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10019 (-1429203781007966208)]} 0 0
[junit4:junit4] 2> 13757 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10019 (-1429203781007966208)]} 0 5
[junit4:junit4] 2> 13760 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10029 (1429203781015306240)]} 0 1
[junit4:junit4] 2> 13767 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10029 (1429203781019500544)]} 0 1
[junit4:junit4] 2> 13767 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10029 (1429203781019500544)]} 0 4
[junit4:junit4] 2> 13792 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[21 (-1429203781049909248)]} 0 0
[junit4:junit4] 2> 13797 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[21 (-1429203781053054976)]} 0 0
[junit4:junit4] 2> 13798 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[21 (-1429203781053054976)]} 0 3
[junit4:junit4] 2> 13802 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[30 (1429203781060395008)]} 0 1
[junit4:junit4] 2> 13807 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10021 (-1429203781065637888)]} 0 0
[junit4:junit4] 2> 13809 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1429203781063540736)]} 0 1
[junit4:junit4] 2> 13809 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[30 (1429203781063540736)]} 0 4
[junit4:junit4] 2> 13812 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10021 (-1429203781068783616)]} 0 0
[junit4:junit4] 2> 13813 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10021 (-1429203781068783616)]} 0 3
[junit4:junit4] 2> 13817 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10030 (1429203781075075072)]} 0 1
[junit4:junit4] 2> 13823 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10030 (1429203781078220800)]} 0 1
[junit4:junit4] 2> 13823 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10030 (1429203781078220800)]} 0 4
[junit4:junit4] 2> 13842 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[22 (-1429203781101289472)]} 0 1
[junit4:junit4] 2> 13846 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[22 (-1429203781104435200)]} 0 0
[junit4:junit4] 2> 13847 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[22 (-1429203781104435200)]} 0 3
[junit4:junit4] 2> 13851 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[31 (1429203781110726656)]} 0 1
[junit4:junit4] 2> 13855 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10022 (-1429203781115969536)]} 0 0
[junit4:junit4] 2> 13858 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1429203781114920960)]} 0 1
[junit4:junit4] 2> 13858 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[31 (1429203781114920960)]} 0 4
[junit4:junit4] 2> 13859 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10022 (-1429203781119115264)]} 0 0
[junit4:junit4] 2> 13860 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10022 (-1429203781119115264)]} 0 2
[junit4:junit4] 2> 13864 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10031 (1429203781124358144)]} 0 1
[junit4:junit4] 2> 13870 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10031 (1429203781127503872)]} 0 1
[junit4:junit4] 2> 13870 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10031 (1429203781127503872)]} 0 4
[junit4:junit4] 2> 13891 T1608 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:48298__collection1&state=recovering&nodeName=127.0.0.1:48298_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 13891 T1576 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 48298
[junit4:junit4] 2> 13892 T1576 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=604822622
[junit4:junit4] 2> 13893 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[32 (1429203781154766848)]} 0 1
[junit4:junit4] 2> 13899 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1429203781157912576)]} 0 1
[junit4:junit4] 2> 13899 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[32 (1429203781157912576)]} 0 4
[junit4:junit4] 2> 13905 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10032 (1429203781167349760)]} 0 1
[junit4:junit4] 2> 13911 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10032 (1429203781170495488)]} 0 1
[junit4:junit4] 2> 13911 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10032 (1429203781170495488)]} 0 4
[junit4:junit4] 2> 13950 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[25 (-1429203781215584256)]} 0 0
[junit4:junit4] 2> 13956 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[25 (-1429203781218729984)]} 0 0
[junit4:junit4] 2> 13957 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[25 (-1429203781218729984)]} 0 4
[junit4:junit4] 2> 13961 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[33 (1429203781226070016)]} 0 1
[junit4:junit4] 2> 13962 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10025 (-1429203781228167168)]} 0 0
[junit4:junit4] 2> 13968 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10025 (-1429203781231312896)]} 0 1
[junit4:junit4] 2> 13968 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1429203781230264320)]} 0 1
[junit4:junit4] 2> 13968 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10025 (-1429203781231312896)]} 0 3
[junit4:junit4] 2> 13969 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[33 (1429203781230264320)]} 0 5
[junit4:junit4] 2> 13972 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10033 (1429203781237604352)]} 0 1
[junit4:junit4] 2> 13974 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[26 (-1429203781240750080)]} 0 0
[junit4:junit4] 2> 13979 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10033 (1429203781241798656)]} 0 1
[junit4:junit4] 2> 13979 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10033 (1429203781241798656)]} 0 4
[junit4:junit4] 2> 13980 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[26 (-1429203781243895808)]} 0 0
[junit4:junit4] 2> 13981 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[26 (-1429203781243895808)]} 0 4
[junit4:junit4] 2> 13985 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10026 (-1429203781252284416)]} 0 0
[junit4:junit4] 2> 13985 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[34 (1429203781251235840)]} 0 1
[junit4:junit4] 2> 13990 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10026 (-1429203781255430144)]} 0 0
[junit4:junit4] 2> 13991 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10026 (-1429203781255430144)]} 0 3
[junit4:junit4] 2> 13992 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1429203781255430145)]} 0 1
[junit4:junit4] 2> 13992 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[34 (1429203781255430145)]} 0 4
[junit4:junit4] 2> 13995 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10034 (1429203781261721600)]} 0 1
[junit4:junit4] 2> 14002 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10034 (1429203781265915904)]} 0 1
[junit4:junit4] 2> 14002 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10034 (1429203781265915904)]} 0 4
[junit4:junit4] 2> 14006 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[27 (-1429203781274304512)]} 0 0
[junit4:junit4] 2> 14011 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[27 (-1429203781277450240)]} 0 0
[junit4:junit4] 2> 14012 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[27 (-1429203781277450240)]} 0 3
[junit4:junit4] 2> 14016 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[35 (1429203781283741696)]} 0 1
[junit4:junit4] 2> 14016 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10027 (-1429203781284790272)]} 0 0
[junit4:junit4] 2> 14021 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10027 (-1429203781287936000)]} 0 0
[junit4:junit4] 2> 14022 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10027 (-1429203781287936000)]} 0 3
[junit4:junit4] 2> 14023 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1429203781287936001)]} 0 1
[junit4:junit4] 2> 14024 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[35 (1429203781287936001)]} 0 5
[junit4:junit4] 2> 14026 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10035 (1429203781294227456)]} 0 1
[junit4:junit4] 2> 14032 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10035 (1429203781297373184)]} 0 1
[junit4:junit4] 2> 14032 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10035 (1429203781297373184)]} 0 4
[junit4:junit4] 2> 14123 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[36 (1429203781395939328)]} 0 1
[junit4:junit4] 2> 14131 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1429203781401182208)]} 0 1
[junit4:junit4] 2> 14132 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10036 (1429203781405376512)]} 0 1
[junit4:junit4] 2> 14132 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[36 (1429203781401182208)]} 0 6
[junit4:junit4] 2> 14139 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10036 (1429203781409570816)]} 0 1
[junit4:junit4] 2> 14139 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10036 (1429203781409570816)]} 0 4
[junit4:junit4] 2> 14179 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[37 (1429203781454659584)]} 0 1
[junit4:junit4] 2> 14185 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1429203781457805312)]} 0 1
[junit4:junit4] 2> 14186 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10037 (1429203781461999616)]} 0 1
[junit4:junit4] 2> 14187 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[37 (1429203781457805312)]} 0 6
[junit4:junit4] 2> 14193 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10037 (1429203781466193920)]} 0 1
[junit4:junit4] 2> 14194 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10037 (1429203781466193920)]} 0 5
[junit4:junit4] 2> 14228 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[28 (-1429203781506039808)]} 0 1
[junit4:junit4] 2> 14233 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[28 (-1429203781510234112)]} 0 0
[junit4:junit4] 2> 14234 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[28 (-1429203781510234112)]} 0 3
[junit4:junit4] 2> 14234 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10028 (-1429203781513379840)]} 0 0
[junit4:junit4] 2> 14238 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[38 (1429203781516525568)]} 0 1
[junit4:junit4] 2> 14240 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10028 (-1429203781516525568)]} 0 0
[junit4:junit4] 2> 14242 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10028 (-1429203781516525568)]} 0 5
[junit4:junit4] 2> 14245 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1429203781520719872)]} 0 1
[junit4:junit4] 2> 14246 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[38 (1429203781520719872)]} 0 5
[junit4:junit4] 2> 14246 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10038 (1429203781524914176)]} 0 1
[junit4:junit4] 2> 14253 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10038 (1429203781529108480)]} 0 1
[junit4:junit4] 2> 14254 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10038 (1429203781529108480)]} 0 5
[junit4:junit4] 2> 14328 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[29 (-1429203781611945984)]} 0 1
[junit4:junit4] 2> 14335 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10029 (-1429203781619286016)]} 0 0
[junit4:junit4] 2> 14335 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[29 (-1429203781616140288)]} 0 0
[junit4:junit4] 2> 14336 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[29 (-1429203781616140288)]} 0 4
[junit4:junit4] 2> 14340 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[39 (1429203781623480320)]} 0 1
[junit4:junit4] 2> 14340 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10029 (-1429203781622431744)]} 0 0
[junit4:junit4] 2> 14341 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10029 (-1429203781622431744)]} 0 3
[junit4:junit4] 2> 14345 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10039 (1429203781628723200)]} 0 1
[junit4:junit4] 2> 14347 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1429203781627674624)]} 0 1
[junit4:junit4] 2> 14347 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[39 (1429203781627674624)]} 0 4
[junit4:junit4] 2> 14351 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10039 (1429203781631868928)]} 0 1
[junit4:junit4] 2> 14351 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10039 (1429203781631868928)]} 0 4
[junit4:junit4] 2> 14373 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[40 (1429203781658083328)]} 0 1
[junit4:junit4] 2> 14377 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10040 (1429203781662277632)]} 0 1
[junit4:junit4] 2> 14379 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1429203781661229056)]} 0 1
[junit4:junit4] 2> 14379 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[40 (1429203781661229056)]} 0 4
[junit4:junit4] 2> 14384 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10040 (1429203781666471936)]} 0 1
[junit4:junit4] 2> 14384 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10040 (1429203781666471936)]} 0 4
[junit4:junit4] 2> 14436 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[31 (-1429203781725192192)]} 0 0
[junit4:junit4] 2> 14441 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10031 (-1429203781730435072)]} 0 0
[junit4:junit4] 2> 14442 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[31 (-1429203781728337920)]} 0 0
[junit4:junit4] 2> 14443 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[31 (-1429203781728337920)]} 0 4
[junit4:junit4] 2> 14446 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10031 (-1429203781733580800)]} 0 0
[junit4:junit4] 2> 14447 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[41 (1429203781735677952)]} 0 1
[junit4:junit4] 2> 14447 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10031 (-1429203781733580800)]} 0 3
[junit4:junit4] 2> 14451 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10041 (1429203781739872256)]} 0 1
[junit4:junit4] 2> 14454 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1429203781739872256)]} 0 1
[junit4:junit4] 2> 14455 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[41 (1429203781739872256)]} 0 5
[junit4:junit4] 2> 14457 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10041 (1429203781743017984)]} 0 1
[junit4:junit4] 2> 14457 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10041 (1429203781743017984)]} 0 4
[junit4:junit4] 2> 14502 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[42 (1429203781793349632)]} 0 1
[junit4:junit4] 2> 14505 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10042 (1429203781796495360)]} 0 1
[junit4:junit4] 2> 14508 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1429203781796495360)]} 0 1
[junit4:junit4] 2> 14509 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[42 (1429203781796495360)]} 0 4
[junit4:junit4] 2> 14511 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10042 (1429203781799641088)]} 0 1
[junit4:junit4] 2> 14511 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10042 (1429203781799641088)]} 0 4
[junit4:junit4] 2> 14525 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[32 (-1429203781818515456)]} 0 0
[junit4:junit4] 2> 14528 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10032 (-1429203781821661184)]} 0 0
[junit4:junit4] 2> 14531 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[32 (-1429203781821661184)]} 0 0
[junit4:junit4] 2> 14532 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[32 (-1429203781821661184)]} 0 4
[junit4:junit4] 2> 14534 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10032 (-1429203781824806912)]} 0 0
[junit4:junit4] 2> 14535 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10032 (-1429203781824806912)]} 0 4
[junit4:junit4] 2> 14536 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[43 (1429203781829001216)]} 0 1
[junit4:junit4] 2> 14539 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10043 (1429203781832146944)]} 0 1
[junit4:junit4] 2> 14543 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1429203781833195520)]} 0 1
[junit4:junit4] 2> 14543 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[43 (1429203781833195520)]} 0 4
[junit4:junit4] 2> 14546 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10043 (1429203781836341248)]} 0 1
[junit4:junit4] 2> 14546 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10043 (1429203781836341248)]} 0 4
[junit4:junit4] 2> 14552 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[34 (-1429203781846827008)]} 0 0
[junit4:junit4] 2> 14555 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10034 (-1429203781849972736)]} 0 0
[junit4:junit4] 2> 14558 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[34 (-1429203781849972736)]} 0 0
[junit4:junit4] 2> 14559 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[34 (-1429203781849972736)]} 0 4
[junit4:junit4] 2> 14561 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10034 (-1429203781853118464)]} 0 0
[junit4:junit4] 2> 14562 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10034 (-1429203781853118464)]} 0 4
[junit4:junit4] 2> 14563 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[44 (1429203781857312768)]} 0 1
[junit4:junit4] 2> 14566 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10044 (1429203781860458496)]} 0 1
[junit4:junit4] 2> 14570 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1429203781861507072)]} 0 1
[junit4:junit4] 2> 14571 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[44 (1429203781861507072)]} 0 5
[junit4:junit4] 2> 14573 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10044 (1429203781864652800)]} 0 1
[junit4:junit4] 2> 14573 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10044 (1429203781864652800)]} 0 4
[junit4:junit4] 2> 14585 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[35 (-1429203781881430016)]} 0 0
[junit4:junit4] 2> 14587 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10035 (-1429203781883527168)]} 0 0
[junit4:junit4] 2> 14590 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[35 (-1429203781884575744)]} 0 0
[junit4:junit4] 2> 14591 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[35 (-1429203781884575744)]} 0 3
[junit4:junit4] 2> 14593 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10035 (-1429203781886672896)]} 0 0
[junit4:junit4] 2> 14594 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10035 (-1429203781886672896)]} 0 4
[junit4:junit4] 2> 14595 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[45 (1429203781890867200)]} 0 1
[junit4:junit4] 2> 14598 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10045 (1429203781894012928)]} 0 1
[junit4:junit4] 2> 14602 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1429203781895061504)]} 0 1
[junit4:junit4] 2> 14602 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[45 (1429203781895061504)]} 0 4
[junit4:junit4] 2> 14605 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10045 (1429203781898207232)]} 0 1
[junit4:junit4] 2> 14605 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10045 (1429203781898207232)]} 0 4
[junit4:junit4] 2> 14677 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[37 (-1429203781976850432)]} 0 1
[junit4:junit4] 2> 14679 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10037 (-1429203781979996160)]} 0 0
[junit4:junit4] 2> 14683 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[37 (-1429203781981044736)]} 0 0
[junit4:junit4] 2> 14684 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[37 (-1429203781981044736)]} 0 4
[junit4:junit4] 2> 14685 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {delete=[10037 (-1429203781983141888)]} 0 0
[junit4:junit4] 2> 14686 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10037 (-1429203781983141888)]} 0 4
[junit4:junit4] 2> 14688 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[46 (1429203781988384768)]} 0 1
[junit4:junit4] 2> 14690 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10046 (1429203781990481920)]} 0 1
[junit4:junit4] 2> 14696 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14696 T1624 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1429203781993627648)]} 0 1
[junit4:junit4] 2> 14697 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:48298__collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:48298_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48298"}
[junit4:junit4] 2> 14697 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[46 (1429203781993627648)]} 0 6
[junit4:junit4] 2> 14697 T1625 C73 P48298 /update {distrib.from=http://127.0.0.1:31693/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10046 (1429203781994676224)]} 0 1
[junit4:junit4] 2> 14698 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10046 (1429203781994676224)]} 0 5
[junit4:junit4] 2> 14708 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14708 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14708 T1634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14708 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14798 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[38 (-1429203782104776704)]} 0 1
[junit4:junit4] 2> 14799 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10038 (-1429203782104776705)]} 0 1
[junit4:junit4] 2> 14801 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[38 (-1429203782107922432)]} 0 0
[junit4:junit4] 2> 14802 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10038 (-1429203782108971008)]} 0 0
[junit4:junit4] 2> 14805 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[47 (1429203782111068160)]} 0 1
[junit4:junit4] 2> 14806 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10047 (1429203782112116736)]} 0 1
[junit4:junit4] 2> 14809 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[47 (1429203782115262464)]} 0 1
[junit4:junit4] 2> 14810 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10047 (1429203782116311040)]} 0 1
[junit4:junit4] 2> 14889 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[39 (-1429203782200197120)]} 0 0
[junit4:junit4] 2> 14890 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10039 (-1429203782201245696)]} 0 0
[junit4:junit4] 2> 14893 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[39 (-1429203782203342848)]} 0 1
[junit4:junit4] 2> 14893 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10039 (-1429203782204391424)]} 0 0
[junit4:junit4] 2> 14895 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[48 (1429203782205440000)]} 0 1
[junit4:junit4] 2> 14897 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10048 (1429203782207537152)]} 0 1
[junit4:junit4] 2> 14899 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[48 (1429203782209634304)]} 0 1
[junit4:junit4] 2> 14900 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10048 (1429203782210682880)]} 0 1
[junit4:junit4] 2> 14919 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[49 (1429203782230605824)]} 0 1
[junit4:junit4] 2> 14920 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10049 (1429203782231654400)]} 0 1
[junit4:junit4] 2> 14922 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[49 (1429203782233751552)]} 0 1
[junit4:junit4] 2> 14924 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10049 (1429203782235848704)]} 0 1
[junit4:junit4] 2> 15001 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[41 (-1429203782317637632)]} 0 0
[junit4:junit4] 2> 15003 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10041 (-1429203782319734784)]} 0 0
[junit4:junit4] 2> 15004 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[41 (-1429203782320783360)]} 0 0
[junit4:junit4] 2> 15006 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10041 (-1429203782322880512)]} 0 0
[junit4:junit4] 2> 15008 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[50 (1429203782323929088)]} 0 1
[junit4:junit4] 2> 15010 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10050 (1429203782326026240)]} 0 1
[junit4:junit4] 2> 15011 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[50 (1429203782327074816)]} 0 1
[junit4:junit4] 2> 15014 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10050 (1429203782330220544)]} 0 1
[junit4:junit4] 2> 15081 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[51 (1429203782400475136)]} 0 1
[junit4:junit4] 2> 15084 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10051 (1429203782403620864)]} 0 1
[junit4:junit4] 2> 15085 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[51 (1429203782404669440)]} 0 1
[junit4:junit4] 2> 15088 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10051 (1429203782407815168)]} 0 1
[junit4:junit4] 2> 15104 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[52 (1429203782424592384)]} 0 1
[junit4:junit4] 2> 15107 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[52 (1429203782427738112)]} 0 1
[junit4:junit4] 2> 15107 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10052 (1429203782427738112)]} 0 1
[junit4:junit4] 2> 15111 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10052 (1429203782431932416)]} 0 1
[junit4:junit4] 2> 15119 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[43 (-1429203782441369600)]} 0 0
[junit4:junit4] 2> 15122 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[43 (-1429203782444515328)]} 0 0
[junit4:junit4] 2> 15123 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10043 (-1429203782445563904)]} 0 0
[junit4:junit4] 2> 15126 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[53 (1429203782447661056)]} 0 1
[junit4:junit4] 2> 15126 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10043 (-1429203782448709632)]} 0 0
[junit4:junit4] 2> 15130 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10053 (1429203782451855360)]} 0 1
[junit4:junit4] 2> 15130 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[53 (1429203782451855360)]} 0 1
[junit4:junit4] 2> 15134 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10053 (1429203782456049664)]} 0 1
[junit4:junit4] 2> 15186 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[48 (-1429203782511624192)]} 0 0
[junit4:junit4] 2> 15189 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[48 (-1429203782514769920)]} 0 0
[junit4:junit4] 2> 15189 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10048 (-1429203782514769920)]} 0 0
[junit4:junit4] 2> 15193 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10048 (-1429203782517915648)]} 0 1
[junit4:junit4] 2> 15193 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[54 (1429203782518964224)]} 0 1
[junit4:junit4] 2> 15197 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10054 (1429203782523158528)]} 0 1
[junit4:junit4] 2> 15198 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[54 (1429203782523158528)]} 0 1
[junit4:junit4] 2> 15201 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10054 (1429203782526304256)]} 0 1
[junit4:junit4] 2> 15243 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[55 (1429203782570344448)]} 0 1
[junit4:junit4] 2> 15245 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10055 (1429203782572441600)]} 0 1
[junit4:junit4] 2> 15246 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[55 (1429203782573490176)]} 0 1
[junit4:junit4] 2> 15248 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10055 (1429203782575587328)]} 0 1
[junit4:junit4] 2> 15308 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[50 (-1429203782639550464)]} 0 0
[junit4:junit4] 2> 15310 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10050 (-1429203782641647616)]} 0 0
[junit4:junit4] 2> 15311 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[50 (-1429203782642696192)]} 0 0
[junit4:junit4] 2> 15314 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10050 (-1429203782644793344)]} 0 1
[junit4:junit4] 2> 15315 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[56 (1429203782645841920)]} 0 1
[junit4:junit4] 2> 15319 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[56 (1429203782650036224)]} 0 1
[junit4:junit4] 2> 15319 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10056 (1429203782650036224)]} 0 1
[junit4:junit4] 2> 15323 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10056 (1429203782654230528)]} 0 1
[junit4:junit4] 2> 15370 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[57 (1429203782703513600)]} 0 1
[junit4:junit4] 2> 15374 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[57 (1429203782707707904)]} 0 1
[junit4:junit4] 2> 15374 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10057 (1429203782707707904)]} 0 1
[junit4:junit4] 2> 15378 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10057 (1429203782711902208)]} 0 1
[junit4:junit4] 2> 15454 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[54 (-1429203782792642560)]} 0 0
[junit4:junit4] 2> 15457 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10054 (-1429203782795788288)]} 0 1
[junit4:junit4] 2> 15458 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[54 (-1429203782796836864)]} 0 0
[junit4:junit4] 2> 15460 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10054 (-1429203782798934016)]} 0 0
[junit4:junit4] 2> 15462 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[58 (1429203782799982592)]} 0 1
[junit4:junit4] 2> 15464 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10058 (1429203782802079744)]} 0 1
[junit4:junit4] 2> 15466 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[58 (1429203782804176896)]} 0 1
[junit4:junit4] 2> 15468 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10058 (1429203782806274048)]} 0 1
[junit4:junit4] 2> 15554 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[59 (1429203782897500160)]} 0 1
[junit4:junit4] 2> 15556 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10059 (1429203782898548736)]} 0 1
[junit4:junit4] 2> 15559 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[59 (1429203782901694464)]} 0 1
[junit4:junit4] 2> 15560 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10059 (1429203782902743040)]} 0 1
[junit4:junit4] 2> 15593 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[55 (-1429203782938394624)]} 0 0
[junit4:junit4] 2> 15594 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10055 (-1429203782939443200)]} 0 0
[junit4:junit4] 2> 15596 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[55 (-1429203782941540352)]} 0 0
[junit4:junit4] 2> 15597 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10055 (-1429203782942588928)]} 0 0
[junit4:junit4] 2> 15600 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[60 (1429203782944686080)]} 0 1
[junit4:junit4] 2> 15601 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10060 (1429203782945734656)]} 0 1
[junit4:junit4] 2> 15604 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[60 (1429203782948880384)]} 0 1
[junit4:junit4] 2> 15605 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10060 (1429203782949928960)]} 0 1
[junit4:junit4] 2> 15659 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[57 (-1429203783007600641)]} 0 1
[junit4:junit4] 2> 15659 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10057 (-1429203783007600640)]} 0 1
[junit4:junit4] 2> 15662 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10057 (-1429203783010746369)]} 0 0
[junit4:junit4] 2> 15662 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[57 (-1429203783010746368)]} 0 0
[junit4:junit4] 2> 15666 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[61 (1429203783013892096)]} 0 1
[junit4:junit4] 2> 15666 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10061 (1429203783013892097)]} 0 1
[junit4:junit4] 2> 15670 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[61 (1429203783018086400)]} 0 1
[junit4:junit4] 2> 15670 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10061 (1429203783018086401)]} 0 1
[junit4:junit4] 2> 15709 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[61 (-1429203783060029440)]} 0 0
[junit4:junit4] 2> 15710 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10061 (-1429203783061078016)]} 0 0
[junit4:junit4] 2> 15712 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[61 (-1429203783063175168)]} 0 0
[junit4:junit4] 2> 15713 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10061 (-1429203783064223744)]} 0 0
[junit4:junit4] 2> 15716 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[62 (1429203783066320896)]} 0 1
[junit4:junit4] 2> 15717 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10062 (1429203783067369472)]} 0 1
[junit4:junit4] 2> 15719 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[62 (1429203783069466624)]} 0 1
[junit4:junit4] 2> 15720 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10062 (1429203783070515200)]} 0 1
[junit4:junit4] 2> 15784 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[63 (1429203783137624064)]} 0 1
[junit4:junit4] 2> 15785 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10063 (1429203783138672640)]} 0 1
[junit4:junit4] 2> 15788 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10063 (1429203783141818368)]} 0 1
[junit4:junit4] 2> 15788 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[63 (1429203783141818369)]} 0 1
[junit4:junit4] 2> 15806 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10064 (1429203783160692736)]} 0 1
[junit4:junit4] 2> 15807 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[64 (1429203783161741312)]} 0 1
[junit4:junit4] 2> 15809 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10064 (1429203783163838464)]} 0 1
[junit4:junit4] 2> 15810 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[64 (1429203783164887040)]} 0 1
[junit4:junit4] 2> 15832 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10065 (1429203783187955712)]} 0 1
[junit4:junit4] 2> 15833 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[65 (1429203783189004288)]} 0 1
[junit4:junit4] 2> 15836 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10065 (1429203783192150016)]} 0 1
[junit4:junit4] 2> 15836 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[65 (1429203783192150017)]} 0 1
[junit4:junit4] 2> C73_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48298_, base_url=http://127.0.0.1:48298}
[junit4:junit4] 2> 15892 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:31693/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 15892 T1637 C73 P48298 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48298 START replicas=[http://127.0.0.1:31693/collection1/] nUpdates=100
[junit4:junit4] 2> 15896 T1576 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15896 T1576 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15897 T1637 C73 P48298 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 15898 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 15898 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 15898 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 15898 T1611 C75 P31693 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 15898 T1637 C73 P48298 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/tlog/tlog.0000000000000000000 refcount=1}}
[junit4:junit4] 2> 15899 T1637 C73 P48298 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:31693/collection1/. core=collection1
[junit4:junit4] 2> 15899 T1576 oasc.RecoveryStrategy.close WARNING Stopping recovery for zkNodeName=127.0.0.1:48298__collection1core=collection1
[junit4:junit4] 2> 15899 T1637 C73 P48298 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 15912 T1612 C75 P31693 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 15923 T1612 C75 P31693 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b24b783 lockFactory=org.apache.lucene.store.NativeFSLockFactory@681dadc7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b24b783 lockFactory=org.apache.lucene.store.NativeFSLockFactory@681dadc7),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, segments_2, _0.fdx, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 15924 T1612 C75 P31693 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, segments_2, _0.fdx, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 15925 T1612 C75 P31693 oass.SolrIndexSearcher.<init> Opening Searcher@7e814e25 realtime
[junit4:junit4] 2> 15926 T1612 C75 P31693 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 15926 T1612 C75 P31693 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 15927 T1637 C73 P48298 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 15927 T1637 C73 P48298 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 15929 T1613 C75 P31693 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 15930 T1613 C75 P31693 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 15930 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 15931 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 15931 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 15931 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10066 (1429203783291764736)]} 0 1
[junit4:junit4] 2> 15932 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[66 (1429203783292813312)]} 0 1
[junit4:junit4] 2> 15933 T1613 C75 P31693 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 15934 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 14
[junit4:junit4] 2> 15935 T1637 C73 P48298 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234221040 forceNew: false
[junit4:junit4] 2> 15935 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Starting download to BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4a3e7760 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fde32eb) fullCopy=true
[junit4:junit4] 2> 15936 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10066 (1429203783295959040)]} 0 2
[junit4:junit4] 2> 15936 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[66 (1429203783295959041)]} 0 2
[junit4:junit4] 2> 15938 T1613 C75 P31693 REQ /replication {file=_0.fnm&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15942 T1613 C75 P31693 REQ /replication {file=_0_Lucene41_0.pos&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15945 T1613 C75 P31693 REQ /replication {file=_0_1.del&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15948 T1613 C75 P31693 REQ /replication {file=_0_dv.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15951 T1613 C75 P31693 REQ /replication {file=_0_nrm.cfs&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15954 T1613 C75 P31693 REQ /replication {file=_0_Lucene41_0.doc&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15957 T1613 C75 P31693 REQ /replication {file=_0_nrm.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15960 T1613 C75 P31693 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15964 T1613 C75 P31693 REQ /replication {file=_0.fdx&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15967 T1613 C75 P31693 REQ /replication {file=_0.si&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15970 T1613 C75 P31693 REQ /replication {file=_0_Lucene41_0.tim&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15973 T1613 C75 P31693 REQ /replication {file=_0_Lucene41_0.tip&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15976 T1613 C75 P31693 REQ /replication {file=_0.fdt&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15979 T1613 C75 P31693 REQ /replication {file=_0_dv.cfe&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 15980 T1637 C73 P48298 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 15980 T1637 C73 P48298 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130311234221040
[junit4:junit4] 2> 15981 T1637 C73 P48298 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 15981 T1637 C73 P48298 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 15981 T1637 C73 P48298 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4] 2> 15983 T1637 C73 P48298 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index/ new=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234221040
[junit4:junit4] 2> 15985 T1637 C73 P48298 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4a3e7760 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fde32eb),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, segments_2, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 15985 T1637 C73 P48298 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, segments_2, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 15986 T1637 C73 P48298 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 15987 T1637 C73 P48298 oass.SolrIndexSearcher.<init> Opening Searcher@281a5a90 main
[junit4:junit4] 2> 15988 T1636 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@281a5a90 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C130/66)}
[junit4:junit4] 2> 15988 T1637 C73 P48298 oash.SnapPuller$2.preClose removing old index files MockDirWrapper(org.apache.lucene.store.RAMDirectory@2260964a lockFactory=org.apache.lucene.store.NativeFSLockFactory@55bdbc3)
[junit4:junit4] 2> 15989 T1637 C73 P48298 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index
[junit4:junit4] 2> 15989 T1637 C73 P48298 oash.SnapPuller$2.postClose removing old index directory MockDirWrapper(org.apache.lucene.store.RAMDirectory@2260964a lockFactory=org.apache.lucene.store.NativeFSLockFactory@55bdbc3)
[junit4:junit4] 2> 15995 T1637 C73 P48298 oasc.RecoveryStrategy.replay Replaying buffered documents. core=collection1
[junit4:junit4] 2> 15995 T1645 C73 P48298 oasu.UpdateLog$LogReplayer.doReplay WARNING Starting log replay tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/tlog/tlog.0000000000000000000 refcount=2} active=true starting pos=5887
[junit4:junit4] 2> 15999 T1645 C73 P48298 oasu.DirectUpdateHandler2.commit start commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 16001 T1645 C73 P48298 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4a3e7760 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fde32eb),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, segments_2, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> commit{dir=BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4a3e7760 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fde32eb),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, segments_3, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 16001 T1645 C73 P48298 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.pos, _0_1.del, _0_dv.cfs, _0_nrm.cfs, _0_Lucene41_0.doc, _0_nrm.cfe, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, segments_3, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 16002 T1645 C73 P48298 oass.SolrIndexSearcher.<init> Opening Searcher@2bb5e4c5 main
[junit4:junit4] 2> 16002 T1645 C73 P48298 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 16003 T1636 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bb5e4c5 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C130/66)}
[junit4:junit4] 2> 16003 T1645 C73 P48298 oasu.UpdateLog$LogReplayer.run WARNING Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=0 positionOfStart=5887}
[junit4:junit4] 2> 16004 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 16004 T1637 C73 P48298 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 16004 T1637 C73 P48298 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16006 T1637 C73 P48298 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 16010 T1576 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@65c3b1c5
[junit4:junit4] 2> 16017 T1576 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=51,cumulative_deletesById=28,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 16018 T1576 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 16018 T1576 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 16019 T1576 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 16021 T1576 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 16021 T1576 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234221040
[junit4:junit4] 2> 16022 T1576 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234221040
[junit4:junit4] 2> 16035 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10063 (-1429203783400816641)]} 0 1
[junit4:junit4] 2> 16035 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[63 (-1429203783400816640)]} 0 1
[junit4:junit4] 2> 16038 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10063 (-1429203783405010944)]} 0 0
[junit4:junit4] 2> 16039 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[63 (-1429203783405010945)]} 0 1
[junit4:junit4] 2> 16042 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[67 (1429203783408156672)]} 0 1
[junit4:junit4] 2> 16042 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10067 (1429203783408156673)]} 0 1
[junit4:junit4] 2> 16046 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[67 (1429203783412350976)]} 0 1
[junit4:junit4] 2> 16046 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10067 (1429203783412350977)]} 0 1
[junit4:junit4] 2> 16112 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[68 (1429203783482605568)]} 0 1
[junit4:junit4] 2> 16113 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10068 (1429203783482605569)]} 0 1
[junit4:junit4] 2> 16116 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[68 (1429203783485751296)]} 0 1
[junit4:junit4] 2> 16117 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10068 (1429203783486799872)]} 0 1
[junit4:junit4] 2> 16120 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[65 (-1429203783489945600)]} 0 1
[junit4:junit4] 2> 16121 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10065 (-1429203783490994176)]} 0 1
[junit4:junit4] 2> 16123 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[65 (-1429203783494139905)]} 0 0
[junit4:junit4] 2> 16123 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10065 (-1429203783494139904)]} 0 0
[junit4:junit4] 2> 16127 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10069 (1429203783497285632)]} 0 1
[junit4:junit4] 2> 16127 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[69 (1429203783497285633)]} 0 1
[junit4:junit4] 2> 16131 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10069 (1429203783501479936)]} 0 1
[junit4:junit4] 2> 16131 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[69 (1429203783501479937)]} 0 1
[junit4:junit4] 2> 16131 T1576 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2
[junit4:junit4] 2> 16132 T1576 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2
[junit4:junit4] 2> 16134 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0008
[junit4:junit4] 2> 16135 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 16135 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:60224 which had sessionid 0x13d58d40e3f0008
[junit4:junit4] 2> 16135 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0008 closed
[junit4:junit4] 2> 16136 T1605 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 16136 T1618 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 16136 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 16137 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 16137 T1576 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 16138 T1598 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 16185 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[66 (-1429203783559151617)]} 0 1
[junit4:junit4] 2> 16185 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10066 (-1429203783559151616)]} 0 1
[junit4:junit4] 2> 16189 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[66 (-1429203783562297344)]} 0 1
[junit4:junit4] 2> 16189 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10066 (-1429203783563345920)]} 0 1
[junit4:junit4] 2> 16193 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10070 (1429203783566491649)]} 0 1
[junit4:junit4] 2> 16193 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[70 (1429203783566491648)]} 0 1
[junit4:junit4] 2> 16197 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[70 (1429203783570685953)]} 0 1
[junit4:junit4] 2> 16197 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10070 (1429203783570685952)]} 0 1
[junit4:junit4] 2> 16212 T1599 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16213 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:48298_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48298"}
[junit4:junit4] 2> 16216 T1599 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:48298_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48298"}
[junit4:junit4] 2> 16229 T1598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 16229 T1618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 16229 T1605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 16263 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10067 (-1429203783640940544)]} 0 1
[junit4:junit4] 2> 16263 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[67 (-1429203783640940545)]} 0 1
[junit4:junit4] 2> 16266 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10067 (-1429203783644086272)]} 0 0
[junit4:junit4] 2> 16267 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[67 (-1429203783644086273)]} 0 1
[junit4:junit4] 2> 16270 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10071 (1429203783647232000)]} 0 1
[junit4:junit4] 2> 16271 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[71 (1429203783648280576)]} 0 1
[junit4:junit4] 2> 16275 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10071 (1429203783652474880)]} 0 1
[junit4:junit4] 2> 16275 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[71 (1429203783652474881)]} 0 1
[junit4:junit4] 2> 16367 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[72 (1429203783749992449)]} 0 1
[junit4:junit4] 2> 16367 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10072 (1429203783749992448)]} 0 1
[junit4:junit4] 2> 16371 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[72 (1429203783753138176)]} 0 1
[junit4:junit4] 2> 16371 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10072 (1429203783753138177)]} 0 1
[junit4:junit4] 2> 16391 T1576 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 16395 T1576 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:48298
[junit4:junit4] 2> 16396 T1576 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 16396 T1576 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 16397 T1576 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572
[junit4:junit4] 2> 16397 T1576 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/solr.xml
[junit4:junit4] 2> 16398 T1576 oasc.CoreContainer.<init> New CoreContainer 1343807993
[junit4:junit4] 2> 16399 T1576 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/'
[junit4:junit4] 2> 16399 T1576 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1362994933572/'
[junit4:junit4] 2> 16420 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[73 (1429203783804518400)]} 0 1
[junit4:junit4] 2> 16420 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10073 (1429203783804518401)]} 0 1
[junit4:junit4] 2> 16424 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[73 (1429203783808712704)]} 0 1
[junit4:junit4] 2> 16424 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10073 (1429203783808712705)]} 0 1
[junit4:junit4] 2> 16462 T1576 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 16463 T1576 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 16463 T1576 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 16464 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 16464 T1576 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 16465 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 16465 T1576 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 16466 T1576 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 16466 T1576 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 16466 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 16488 T1576 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 16488 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[68 (-1429203783876870144)]} 0 0
[junit4:junit4] 2> 16489 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10068 (-1429203783877918720)]} 0 0
[junit4:junit4] 2> 16491 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[68 (-1429203783880015872)]} 0 0
[junit4:junit4] 2> 16493 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10068 (-1429203783882113024)]} 0 0
[junit4:junit4] 2> 16495 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[74 (1429203783883161600)]} 0 1
[junit4:junit4] 2> 16497 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10074 (1429203783885258752)]} 0 1
[junit4:junit4] 2> 16499 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[74 (1429203783887355904)]} 0 1
[junit4:junit4] 2> 16500 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10074 (1429203783888404480)]} 0 1
[junit4:junit4] 2> 16508 T1576 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:31684/solr
[junit4:junit4] 2> 16509 T1576 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 16509 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@704ddd1e
[junit4:junit4] 2> 16510 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16511 T1655 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16511 T1655 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 16512 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44715
[junit4:junit4] 2> 16512 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44715
[junit4:junit4] 2> 16513 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f0009 with negotiated timeout 20000 for client /140.211.11.196:44715
[junit4:junit4] 2> 16513 T1655 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f0009, negotiated timeout = 20000
[junit4:junit4] 2> 16514 T1656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@704ddd1e name:ZooKeeperConnection Watcher:127.0.0.1:31684 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16514 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16515 T1581 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d58d40e3f0009
[junit4:junit4] 2> 16516 T1656 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 16516 T1578 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:44715 which had sessionid 0x13d58d40e3f0009
[junit4:junit4] 2> 16516 T1576 oaz.ZooKeeper.close Session: 0x13d58d40e3f0009 closed
[junit4:junit4] 2> 16516 T1576 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 16528 T1576 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:31684/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1104522
[junit4:junit4] 2> 16529 T1576 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16529 T1657 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:31684. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16530 T1657 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:31684, initiating session
[junit4:junit4] 2> 16530 T1578 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44716
[junit4:junit4] 2> 16530 T1578 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44716
[junit4:junit4] 2> 16531 T1580 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d58d40e3f000a with negotiated timeout 20000 for client /140.211.11.196:44716
[junit4:junit4] 2> 16531 T1657 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:31684, sessionid = 0x13d58d40e3f000a, negotiated timeout = 20000
[junit4:junit4] 2> 16532 T1658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1104522 name:ZooKeeperConnection Watcher:127.0.0.1:31684/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16532 T1576 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16533 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f000a type:create cxid:0x1 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16534 T1581 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d58d40e3f000a type:create cxid:0x2 zxid:0x95 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16543 T1576 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 16597 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[70 (-1429203783991164928)]} 0 0
[junit4:junit4] 2> 16598 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10070 (-1429203783992213504)]} 0 0
[junit4:junit4] 2> 16602 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10070 (-1429203783995359232)]} 0 0
[junit4:junit4] 2> 16602 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[70 (-1429203783995359233)]} 0 0
[junit4:junit4] 2> 16605 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10075 (1429203783998504960)]} 0 1
[junit4:junit4] 2> 16605 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[75 (1429203783998504961)]} 0 1
[junit4:junit4] 2> 16609 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10075 (1429203784002699264)]} 0 1
[junit4:junit4] 2> 16609 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[75 (1429203784002699265)]} 0 1
[junit4:junit4] 2> 16646 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10076 (1429203784041496576)]} 0 1
[junit4:junit4] 2> 16647 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[76 (1429203784042545152)]} 0 1
[junit4:junit4] 2> 16650 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[76 (1429203784045690881)]} 0 1
[junit4:junit4] 2> 16650 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10076 (1429203784045690880)]} 0 1
[junit4:junit4] 2> 16684 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[71 (-1429203784082391040)]} 0 0
[junit4:junit4] 2> 16684 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10071 (-1429203784082391041)]} 0 0
[junit4:junit4] 2> 16687 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[71 (-1429203784085536768)]} 0 0
[junit4:junit4] 2> 16687 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10071 (-1429203784085536769)]} 0 0
[junit4:junit4] 2> 16691 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10077 (1429203784088682496)]} 0 1
[junit4:junit4] 2> 16691 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[77 (1429203784088682497)]} 0 1
[junit4:junit4] 2> 16695 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10077 (1429203784092876800)]} 0 1
[junit4:junit4] 2> 16695 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[77 (1429203784092876801)]} 0 1
[junit4:junit4] 2> 16747 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10078 (1429203784147402752)]} 0 1
[junit4:junit4] 2> 16748 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[78 (1429203784148451328)]} 0 1
[junit4:junit4] 2> 16751 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[78 (1429203784151597056)]} 0 1
[junit4:junit4] 2> 16751 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10078 (1429203784151597057)]} 0 1
[junit4:junit4] 2> 16844 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[79 (1429203784249114624)]} 0 1
[junit4:junit4] 2> 16845 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10079 (1429203784251211776)]} 0 1
[junit4:junit4] 2> 16848 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[79 (1429203784253308928)]} 0 1
[junit4:junit4] 2> 16849 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10079 (1429203784254357504)]} 0 1
[junit4:junit4] 2> 16860 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[72 (-1429203784266940416)]} 0 0
[junit4:junit4] 2> 16860 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10072 (-1429203784266940417)]} 0 0
[junit4:junit4] 2> 16863 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10072 (-1429203784270086144)]} 0 0
[junit4:junit4] 2> 16863 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[72 (-1429203784270086145)]} 0 0
[junit4:junit4] 2> 16867 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[80 (1429203784273231873)]} 0 1
[junit4:junit4] 2> 16867 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10080 (1429203784273231872)]} 0 1
[junit4:junit4] 2> 16871 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10080 (1429203784277426176)]} 0 1
[junit4:junit4] 2> 16871 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[80 (1429203784277426177)]} 0 1
[junit4:junit4] 2> 16891 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[10074 (-1429203784299446272)]} 0 0
[junit4:junit4] 2> 16892 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[74 (-1429203784300494848)]} 0 0
[junit4:junit4] 2> 16894 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[10074 (-1429203784302592000)]} 0 0
[junit4:junit4] 2> 16895 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[74 (-1429203784303640576)]} 0 0
[junit4:junit4] 2> 16898 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10081 (1429203784305737728)]} 0 1
[junit4:junit4] 2> 16899 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[81 (1429203784306786304)]} 0 1
[junit4:junit4] 2> 16901 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10081 (1429203784308883456)]} 0 1
[junit4:junit4] 2> 16903 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[81 (1429203784310980608)]} 0 1
[junit4:junit4] 2> 16928 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10082 (1429203784337195008)]} 0 1
[junit4:junit4] 2> 16930 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[82 (1429203784339292160)]} 0 1
[junit4:junit4] 2> 16931 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10082 (1429203784340340736)]} 0 1
[junit4:junit4] 2> 16933 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[82 (1429203784342437888)]} 0 1
[junit4:junit4] 2> 17029 T1589 C74 P31687 /update {wt=javabin&version=2} {delete=[10076 (-1429203784444149760)]} 0 0
[junit4:junit4] 2> 17031 T1588 C74 P31687 /update {wt=javabin&version=2} {delete=[76 (-1429203784446246912)]} 0 0
[junit4:junit4] 2> 17032 T1609 C75 P31693 /update {wt=javabin&version=2} {delete=[10076 (-1429203784447295488)]} 0 0
[junit4:junit4] 2> 17034 T1610 C75 P31693 /update {wt=javabin&version=2} {delete=[76 (-1429203784449392640)]} 0 0
[junit4:junit4] 2> 17036 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10083 (1429203784450441216)]} 0 1
[junit4:junit4] 2> 17038 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[83 (1429203784452538368)]} 0 1
[junit4:junit4] 2> 17040 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10083 (1429203784454635520)]} 0 1
[junit4:junit4] 2> 17041 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[83 (1429203784455684096)]} 0 1
[junit4:junit4] 2> 17130 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10084 (1429203784549007360)]} 0 1
[junit4:junit4] 2> 17131 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[84 (1429203784550055936)]} 0 1
[junit4:junit4] 2> 17133 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10084 (1429203784552153088)]} 0 1
[junit4:junit4] 2> 17135 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[84 (1429203784554250240)]} 0 1
[junit4:junit4] 2> 17211 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10085 (1429203784633942016)]} 0 1
[junit4:junit4] 2> 17213 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[85 (1429203784636039168)]} 0 1
[junit4:junit4] 2> 17215 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10085 (1429203784638136320)]} 0 1
[junit4:junit4] 2> 17216 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[85 (1429203784639184896)]} 0 1
[junit4:junit4] 2> 17314 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[86 (1429203784741945344)]} 0 1
[junit4:junit4] 2> 17314 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10086 (1429203784741945345)]} 0 1
[junit4:junit4] 2> 17317 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10086 (1429203784745091072)]} 0 1
[junit4:junit4] 2> 17317 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[86 (1429203784745091073)]} 0 1
[junit4:junit4] 2> 17381 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[10087 (1429203784812199936)]} 0 1
[junit4:junit4] 2> 17382 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[87 (1429203784813248512)]} 0 1
[junit4:junit4] 2> 17384 T1610 C75 P31693 /update {wt=javabin&version=2} {add=[10087 (1429203784815345664)]} 0 1
[junit4:junit4] 2> 17385 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[87 (1429203784816394240)]} 0 1
[junit4:junit4] 2> 17432 T1589 C74 P31687 /update {wt=javabin&version=2} {add=[10088 (1429203784865677312)]} 0 1
[junit4:junit4] 2> 17433 T1588 C74 P31687 /update {wt=javabin&version=2} {add=[88 (1429203784866725888)]} 0 1
[junit4:junit4] 2> 17436 T1634 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 17436 T1609 C75 P31693 /update {wt=javabin&version=2} {add=[10088 (1429203784869871616)]} 0 1
[junit4:junit4] 2> 17436 T1634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live node
[...truncated too long message...]
y=0,cumulative_errors=0}
[junit4:junit4] 2> 64409 T1658 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 64409 T1658 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 64409 T1658 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 64410 T1658 C76 P48298 oasu.TransactionLog.forceClose SEVERE Error: Forcing close of tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/tlog/tlog.0000000000000000000 refcount=1}
[junit4:junit4] 2> 64412 T1658 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 64413 T1658 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234228698
[junit4:junit4] 2> 64413 T1658 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2/index.20130311234228698
[junit4:junit4] 2> 64651 T1658 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2
[junit4:junit4] 2> 64652 T1658 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1362994925111/jetty2
[junit4:junit4] 2> 64652 T1658 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 64653 T1658 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 64653 T1658 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 64654 T1658 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 64654 T1658 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=pt_PT, timezone=Pacific/Fakaofo
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=325360640,total=424673280
[junit4:junit4] 2> NOTE: All tests run in this JVM: [DocValuesTest, PluginInfoTest, SynonymTokenizerTest, SoftAutoCommitTest, TestPseudoReturnFields, TestPluginEnable, UUIDFieldTest, MBeansHandlerTest, TestDocumentBuilder, SolrIndexSplitterTest, PrimitiveFieldTypeTest, DateFieldTest, SliceStateTest, TestCharFilters, TestFastWriter, TestGroupingSearch, TestSearchPerf, LeaderElectionTest, TestRandomDVFaceting, FieldMutatingUpdateProcessorTest, BasicDistributedZkTest, SolrCoreTest, AlternateDirectoryTest, TermVectorComponentTest, CircularListTest, SuggesterFSTTest, TestLRUCache, SimpleFacetsTest, TestExtendedDismaxParser, TestRealTimeGet, SearchHandlerTest, ClusterStateUpdateTest, BasicFunctionalityTest, TestXIncludeConfig, ResponseLogComponentTest, TestBadConfig, SyncSliceTest, ShardRoutingTest, TestIBSimilarityFactory, TestFastLRUCache, PrimUtilsTest, RecoveryZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RecoveryZkTest -Dtests.seed=97FCCAC348BCDC9B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pt_PT -Dtests.timezone=Pacific/Fakaofo -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J0 | RecoveryZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=1659, name=coreLoadExecutor-710-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([97FCCAC348BCDC9B]:0)
[junit4:junit4] > at java.lang.Thread.getStackTrace(Thread.java:1567)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:980)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:878)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:870)
[junit4:junit4] > at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:34)
[junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
[junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
[junit4:junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244)
[junit4:junit4] > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458)
[junit4:junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:744)
[junit4:junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
[junit4:junit4] > at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:874)
[junit4:junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:957)
[junit4:junit4] > at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:580)
[junit4:junit4] > at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:575)
[junit4:junit4] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4:junit4] > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > Caused by: java.lang.AssertionError: Directory not closed: BaseDirectoryWrapper(org.apache.lucene.store.RAMDirectory@674f59da lockFactory=org.apache.lucene.store.NativeFSLockFactory@5e1cdb2)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > ... 1 more
[junit4:junit4] Completed on J0 in 64.73s, 1 test, 1 error <<< FAILURES!
[...truncated 643 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 270 suites, 1138 tests, 1 suite-level error, 27 ignored (8 assumptions)
Total time: 51 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3799 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3799/
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
No registered leader was found, collection:collection1 slice:shard1
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found, collection:collection1 slice:shard1
at __randomizedtesting.SeedInfo.seed([241FB552655FD2FE:A5F93B4A1200B2C2]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:481)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:295)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:116)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9043 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T1446 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 5 T1446 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664
[junit4:junit4] 2> 6 T1446 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 7 T1447 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 10 T1447 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 10 T1447 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 10 T1447 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 11 T1447 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 12 T1447 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 108 T1446 oasc.ZkTestServer.run start zk server on port:11124
[junit4:junit4] 2> 108 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@543a16f3
[junit4:junit4] 2> 109 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 109 T1452 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 110 T1452 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 110 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11125
[junit4:junit4] 2> 111 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11125
[junit4:junit4] 2> 111 T1450 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 116 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870000 with negotiated timeout 10000 for client /140.211.11.196:11125
[junit4:junit4] 2> 116 T1452 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870000, negotiated timeout = 10000
[junit4:junit4] 2> 116 T1453 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@543a16f3 name:ZooKeeperConnection Watcher:127.0.0.1:11124 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 116 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 117 T1446 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 125 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870000
[junit4:junit4] 2> 133 T1453 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 133 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11125 which had sessionid 0x13d5567e7870000
[junit4:junit4] 2> 133 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870000 closed
[junit4:junit4] 2> 134 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a3cfbe9
[junit4:junit4] 2> 134 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 135 T1454 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 135 T1454 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 135 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11126
[junit4:junit4] 2> 136 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11126
[junit4:junit4] 2> 137 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870001 with negotiated timeout 10000 for client /140.211.11.196:11126
[junit4:junit4] 2> 137 T1454 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870001, negotiated timeout = 10000
[junit4:junit4] 2> 138 T1455 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a3cfbe9 name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 138 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 138 T1446 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 142 T1446 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 145 T1446 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 153 T1446 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 156 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 157 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 169 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 169 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 275 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 276 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 279 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 280 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 288 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 289 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 292 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 293 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 296 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 297 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 300 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 301 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 304 T1446 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 305 T1446 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 327 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870001
[junit4:junit4] 2> 328 T1455 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 328 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11126 which had sessionid 0x13d5567e7870001
[junit4:junit4] 2> 328 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870001 closed
[junit4:junit4] 2> 518 T1446 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 522 T1446 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11127
[junit4:junit4] 2> 523 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 524 T1446 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 524 T1446 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987
[junit4:junit4] 2> 525 T1446 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/solr.xml
[junit4:junit4] 2> 525 T1446 oasc.CoreContainer.<init> New CoreContainer 165339111
[junit4:junit4] 2> 526 T1446 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/'
[junit4:junit4] 2> 526 T1446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/'
[junit4:junit4] 2> 590 T1446 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 591 T1446 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 591 T1446 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 592 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 592 T1446 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 593 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 593 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 594 T1446 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 594 T1446 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 595 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 617 T1446 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 638 T1446 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11124/solr
[junit4:junit4] 2> 638 T1446 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 639 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@70ac2c69
[junit4:junit4] 2> 640 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 640 T1465 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 641 T1465 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 642 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11128
[junit4:junit4] 2> 642 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11128
[junit4:junit4] 2> 644 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870002 with negotiated timeout 20000 for client /140.211.11.196:11128
[junit4:junit4] 2> 644 T1465 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870002, negotiated timeout = 20000
[junit4:junit4] 2> 645 T1466 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70ac2c69 name:ZooKeeperConnection Watcher:127.0.0.1:11124 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 645 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 647 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870002
[junit4:junit4] 2> 654 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11128 which had sessionid 0x13d5567e7870002
[junit4:junit4] 2> 654 T1466 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 654 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870002 closed
[junit4:junit4] 2> 655 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 667 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@255f0fed
[junit4:junit4] 2> 669 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 669 T1467 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 670 T1467 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 670 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11129
[junit4:junit4] 2> 670 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11129
[junit4:junit4] 2> 671 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870003 with negotiated timeout 20000 for client /140.211.11.196:11129
[junit4:junit4] 2> 671 T1467 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870003, negotiated timeout = 20000
[junit4:junit4] 2> 672 T1468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@255f0fed name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 672 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 675 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 683 T1446 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 685 T1446 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11127_
[junit4:junit4] 2> 686 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:11127_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:11127_
[junit4:junit4] 2> 687 T1446 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11127_
[junit4:junit4] 2> 691 T1446 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 707 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 708 T1446 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 711 T1446 oasc.Overseer.start Overseer (id=89321472371654659-127.0.0.1:11127_-n_0000000000) starting
[junit4:junit4] 2> 712 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 719 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 721 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 722 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 731 T1470 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 732 T1446 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 736 T1446 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 745 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 748 T1469 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 752 T1471 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/collection1
[junit4:junit4] 2> 752 T1471 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 753 T1471 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 753 T1471 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 755 T1471 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/collection1/'
[junit4:junit4] 2> 756 T1471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 757 T1471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/collection1/lib/README' to classloader
[junit4:junit4] 2> 809 T1471 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 873 T1471 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 974 T1471 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 987 T1471 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1663 T1471 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1676 T1471 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1679 T1471 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1691 T1471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1696 T1471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1699 T1471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1700 T1471 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1701 T1471 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1701 T1471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1702 T1471 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1702 T1471 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1702 T1471 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362937505987/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data/
[junit4:junit4] 2> 1702 T1471 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ad6dbe
[junit4:junit4] 2> 1703 T1471 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1703 T1471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data forceNew: false
[junit4:junit4] 2> 1704 T1471 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data/index/
[junit4:junit4] 2> 1704 T1471 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1705 T1471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data/index forceNew: false
[junit4:junit4] 2> 1707 T1471 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1707 T1471 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1709 T1471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1709 T1471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1710 T1471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1710 T1471 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1711 T1471 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1711 T1471 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1711 T1471 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1711 T1471 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1712 T1471 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1720 T1471 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1725 T1471 oass.SolrIndexSearcher.<init> Opening Searcher@7cf80c02 main
[junit4:junit4] 2> 1726 T1471 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/control/data/tlog
[junit4:junit4] 2> 1726 T1471 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1726 T1471 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1729 T1472 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7cf80c02 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1730 T1471 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1730 T1471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1731 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 2251 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2252 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:11127_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11127"}
[junit4:junit4] 2> 2252 T1469 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2253 T1469 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 2253 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 2268 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2772 T1471 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2772 T1471 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11127 collection:control_collection shard:shard1
[junit4:junit4] 2> 2773 T1471 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2789 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2790 T1471 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2791 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2800 T1471 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2801 T1471 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2801 T1471 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11127/collection1/
[junit4:junit4] 2> 2801 T1471 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2801 T1471 oasc.SyncStrategy.syncToMe http://127.0.0.1:11127/collection1/ has no replicas
[junit4:junit4] 2> 2802 T1471 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11127/collection1/
[junit4:junit4] 2> 2802 T1471 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2806 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3773 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3825 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3879 T1471 oasc.ZkController.register We are http://127.0.0.1:11127/collection1/ and leader is http://127.0.0.1:11127/collection1/
[junit4:junit4] 2> 3879 T1471 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11127
[junit4:junit4] 2> 3879 T1471 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3879 T1471 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3880 T1471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3881 T1471 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3883 T1446 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 3883 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3884 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3898 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3899 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6110fd5e
[junit4:junit4] 2> 3900 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3900 T1474 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3901 T1474 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 3901 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11131
[junit4:junit4] 2> 3902 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11131
[junit4:junit4] 2> 3903 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870004 with negotiated timeout 10000 for client /140.211.11.196:11131
[junit4:junit4] 2> 3903 T1474 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870004, negotiated timeout = 10000
[junit4:junit4] 2> 3903 T1475 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6110fd5e name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3903 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3905 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3907 T1446 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4105 T1446 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4108 T1446 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11133
[junit4:junit4] 2> 4109 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4110 T1446 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4110 T1446 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566
[junit4:junit4] 2> 4111 T1446 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/solr.xml
[junit4:junit4] 2> 4111 T1446 oasc.CoreContainer.<init> New CoreContainer 772374618
[junit4:junit4] 2> 4112 T1446 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/'
[junit4:junit4] 2> 4113 T1446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/'
[junit4:junit4] 2> 4181 T1446 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4182 T1446 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4182 T1446 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4183 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4183 T1446 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4184 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4184 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4185 T1446 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4185 T1446 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4186 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4207 T1446 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4228 T1446 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11124/solr
[junit4:junit4] 2> 4229 T1446 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4229 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@182608de
[junit4:junit4] 2> 4230 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4231 T1485 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4232 T1485 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 4232 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11134
[junit4:junit4] 2> 4232 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11134
[junit4:junit4] 2> 4235 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870005 with negotiated timeout 20000 for client /140.211.11.196:11134
[junit4:junit4] 2> 4236 T1485 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870005, negotiated timeout = 20000
[junit4:junit4] 2> 4236 T1486 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@182608de name:ZooKeeperConnection Watcher:127.0.0.1:11124 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4236 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4238 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870005
[junit4:junit4] 2> 4239 T1486 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4239 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11134 which had sessionid 0x13d5567e7870005
[junit4:junit4] 2> 4239 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870005 closed
[junit4:junit4] 2> 4240 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4250 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11a777d6
[junit4:junit4] 2> 4251 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4251 T1487 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4252 T1487 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 4252 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11135
[junit4:junit4] 2> 4252 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11135
[junit4:junit4] 2> 4254 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870006 with negotiated timeout 20000 for client /140.211.11.196:11135
[junit4:junit4] 2> 4254 T1487 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870006, negotiated timeout = 20000
[junit4:junit4] 2> 4254 T1488 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11a777d6 name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4254 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4255 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4257 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4264 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5268 T1446 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11133_
[junit4:junit4] 2> 5270 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:11133_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:11133_
[junit4:junit4] 2> 5271 T1446 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11133_
[junit4:junit4] 2> 5275 T1468 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5275 T1475 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5275 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5275 T1488 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5293 T1489 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/collection1
[junit4:junit4] 2> 5293 T1489 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 5293 T1489 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 5294 T1489 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 5295 T1489 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/collection1/'
[junit4:junit4] 2> 5296 T1489 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/collection1/lib/README' to classloader
[junit4:junit4] 2> 5297 T1489 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 5339 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 5340 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:11127_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11127"}
[junit4:junit4] 2> 5348 T1489 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 5350 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5350 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5350 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5411 T1489 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5512 T1489 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5523 T1489 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6184 T1489 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6194 T1489 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6197 T1489 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6213 T1489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6217 T1489 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6221 T1489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6223 T1489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6223 T1489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6223 T1489 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6225 T1489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6225 T1489 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6225 T1489 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1/
[junit4:junit4] 2> 6225 T1489 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ad6dbe
[junit4:junit4] 2> 6226 T1489 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6226 T1489 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1 forceNew: false
[junit4:junit4] 2> 6226 T1489 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1/index/
[junit4:junit4] 2> 6227 T1489 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6227 T1489 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1/index forceNew: false
[junit4:junit4] 2> 6229 T1489 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6230 T1489 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6232 T1489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6232 T1489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6232 T1489 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6233 T1489 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6233 T1489 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6234 T1489 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6234 T1489 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6234 T1489 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6235 T1489 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6245 T1489 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6250 T1489 oass.SolrIndexSearcher.<init> Opening Searcher@1ab2e368 main
[junit4:junit4] 2> 6250 T1489 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty1/tlog
[junit4:junit4] 2> 6251 T1489 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6251 T1489 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6254 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ab2e368 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6256 T1489 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 6256 T1489 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6859 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6860 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11133_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11133"}
[junit4:junit4] 2> 6860 T1469 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6860 T1469 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6872 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6872 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6872 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7259 T1489 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7259 T1489 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11133 collection:collection1 shard:shard1
[junit4:junit4] 2> 7260 T1489 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7276 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7276 T1489 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7277 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7279 T1489 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7279 T1489 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7279 T1489 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11133/collection1/
[junit4:junit4] 2> 7280 T1489 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7280 T1489 oasc.SyncStrategy.syncToMe http://127.0.0.1:11133/collection1/ has no replicas
[junit4:junit4] 2> 7280 T1489 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11133/collection1/
[junit4:junit4] 2> 7280 T1489 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7285 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8377 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8393 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8393 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8393 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8409 T1489 oasc.ZkController.register We are http://127.0.0.1:11133/collection1/ and leader is http://127.0.0.1:11133/collection1/
[junit4:junit4] 2> 8409 T1489 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11133
[junit4:junit4] 2> 8409 T1489 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8409 T1489 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8410 T1489 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8411 T1489 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8413 T1446 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 8413 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8414 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8654 T1446 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8657 T1446 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11140
[junit4:junit4] 2> 8658 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8659 T1446 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8659 T1446 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087
[junit4:junit4] 2> 8660 T1446 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/solr.xml
[junit4:junit4] 2> 8660 T1446 oasc.CoreContainer.<init> New CoreContainer 1580241500
[junit4:junit4] 2> 8661 T1446 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/'
[junit4:junit4] 2> 8662 T1446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/'
[junit4:junit4] 2> 8731 T1446 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8732 T1446 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8732 T1446 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8733 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8733 T1446 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8734 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8734 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8735 T1446 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8735 T1446 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8736 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8756 T1446 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8779 T1446 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11124/solr
[junit4:junit4] 2> 8780 T1446 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8781 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@aae0746
[junit4:junit4] 2> 8782 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8782 T1501 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8783 T1501 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 8784 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11141
[junit4:junit4] 2> 8784 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11141
[junit4:junit4] 2> 8786 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870007 with negotiated timeout 20000 for client /140.211.11.196:11141
[junit4:junit4] 2> 8786 T1501 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870007, negotiated timeout = 20000
[junit4:junit4] 2> 8787 T1502 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aae0746 name:ZooKeeperConnection Watcher:127.0.0.1:11124 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8787 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8788 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870007
[junit4:junit4] 2> 8790 T1502 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8790 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11141 which had sessionid 0x13d5567e7870007
[junit4:junit4] 2> 8790 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870007 closed
[junit4:junit4] 2> 8791 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8801 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@77c1d334
[junit4:junit4] 2> 8802 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8802 T1503 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8803 T1503 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 8803 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11142
[junit4:junit4] 2> 8803 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11142
[junit4:junit4] 2> 8805 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870008 with negotiated timeout 20000 for client /140.211.11.196:11142
[junit4:junit4] 2> 8805 T1503 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870008, negotiated timeout = 20000
[junit4:junit4] 2> 8805 T1504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77c1d334 name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8805 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8806 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8808 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8817 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9820 T1446 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11140_
[junit4:junit4] 2> 9822 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:11140_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:11140_
[junit4:junit4] 2> 9823 T1446 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11140_
[junit4:junit4] 2> 9826 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9826 T1468 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9827 T1488 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9827 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9827 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9827 T1504 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9828 T1475 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9846 T1505 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/collection1
[junit4:junit4] 2> 9846 T1505 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9847 T1505 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9847 T1505 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9849 T1505 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/collection1/'
[junit4:junit4] 2> 9850 T1505 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/collection1/lib/README' to classloader
[junit4:junit4] 2> 9850 T1505 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9901 T1505 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 9907 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9908 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11133_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11133"}
[junit4:junit4] 2> 9913 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9913 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9914 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9913 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9963 T1505 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 10064 T1505 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 10073 T1505 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 10707 T1505 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 10715 T1505 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 10718 T1505 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 10729 T1505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10733 T1505 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10736 T1505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10737 T1505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10737 T1505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10737 T1505 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10738 T1505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10738 T1505 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10739 T1505 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362937514087/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2/
[junit4:junit4] 2> 10739 T1505 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ad6dbe
[junit4:junit4] 2> 10739 T1505 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10740 T1505 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2 forceNew: false
[junit4:junit4] 2> 10740 T1505 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2/index/
[junit4:junit4] 2> 10741 T1505 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 10741 T1505 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2/index forceNew: false
[junit4:junit4] 2> 10743 T1505 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 10743 T1505 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 10745 T1505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 10745 T1505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 10746 T1505 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 10746 T1505 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 10747 T1505 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 10747 T1505 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 10747 T1505 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 10748 T1505 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 10748 T1505 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 10756 T1505 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 10760 T1505 oass.SolrIndexSearcher.<init> Opening Searcher@af25d95 main
[junit4:junit4] 2> 10761 T1505 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty2/tlog
[junit4:junit4] 2> 10761 T1505 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 10762 T1505 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 10765 T1506 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@af25d95 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 10766 T1505 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 10767 T1505 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11425 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11426 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11140_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11140"}
[junit4:junit4] 2> 11426 T1469 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11426 T1469 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11466 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11466 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11466 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11466 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11795 T1505 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11795 T1505 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11140 collection:collection1 shard:shard2
[junit4:junit4] 2> 11796 T1505 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 11804 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:delete cxid:0x47 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 11812 T1505 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11813 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:create cxid:0x48 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11816 T1505 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11816 T1505 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11817 T1505 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11140/collection1/
[junit4:junit4] 2> 11817 T1505 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11817 T1505 oasc.SyncStrategy.syncToMe http://127.0.0.1:11140/collection1/ has no replicas
[junit4:junit4] 2> 11817 T1505 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11140/collection1/
[junit4:junit4] 2> 11818 T1505 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 11821 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870008 type:create cxid:0x51 zxid:0x83 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12971 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12990 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12990 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12990 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12990 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13003 T1505 oasc.ZkController.register We are http://127.0.0.1:11140/collection1/ and leader is http://127.0.0.1:11140/collection1/
[junit4:junit4] 2> 13003 T1505 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11140
[junit4:junit4] 2> 13003 T1505 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 13003 T1505 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 13004 T1505 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13005 T1505 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13006 T1446 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 13007 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 13008 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 13218 T1446 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 13222 T1446 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11146
[junit4:junit4] 2> 13223 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 13223 T1446 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 13224 T1446 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682
[junit4:junit4] 2> 13224 T1446 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/solr.xml
[junit4:junit4] 2> 13225 T1446 oasc.CoreContainer.<init> New CoreContainer 1012634430
[junit4:junit4] 2> 13226 T1446 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/'
[junit4:junit4] 2> 13226 T1446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/'
[junit4:junit4] 2> 13289 T1446 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 13290 T1446 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 13290 T1446 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 13291 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 13291 T1446 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 13292 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 13292 T1446 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 13293 T1446 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 13293 T1446 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 13294 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 13316 T1446 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 13336 T1446 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11124/solr
[junit4:junit4] 2> 13337 T1446 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 13338 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3eb53fd3
[junit4:junit4] 2> 13339 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13339 T1517 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13340 T1517 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 13340 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11147
[junit4:junit4] 2> 13341 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11147
[junit4:junit4] 2> 13343 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e7870009 with negotiated timeout 20000 for client /140.211.11.196:11147
[junit4:junit4] 2> 13343 T1517 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e7870009, negotiated timeout = 20000
[junit4:junit4] 2> 13343 T1518 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eb53fd3 name:ZooKeeperConnection Watcher:127.0.0.1:11124 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13344 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13345 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e7870009
[junit4:junit4] 2> 13346 T1518 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 13346 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11147 which had sessionid 0x13d5567e7870009
[junit4:junit4] 2> 13346 T1446 oaz.ZooKeeper.close Session: 0x13d5567e7870009 closed
[junit4:junit4] 2> 13347 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 13359 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@24cad7eb
[junit4:junit4] 2> 13360 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13360 T1519 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13361 T1519 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 13361 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11148
[junit4:junit4] 2> 13361 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11148
[junit4:junit4] 2> 13363 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e787000a with negotiated timeout 20000 for client /140.211.11.196:11148
[junit4:junit4] 2> 13363 T1519 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e787000a, negotiated timeout = 20000
[junit4:junit4] 2> 13363 T1520 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24cad7eb name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13363 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13364 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e787000a type:create cxid:0x1 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13366 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e787000a type:create cxid:0x2 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13375 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 14378 T1446 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11146_
[junit4:junit4] 2> 14379 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e787000a type:delete cxid:0xb zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:11146_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:11146_
[junit4:junit4] 2> 14405 T1446 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11146_
[junit4:junit4] 2> 14408 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14409 T1504 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14409 T1520 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14409 T1468 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14409 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14409 T1488 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14410 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14410 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14410 T1475 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14421 T1521 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/collection1
[junit4:junit4] 2> 14422 T1521 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 14422 T1521 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 14422 T1521 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 14424 T1521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/collection1/'
[junit4:junit4] 2> 14425 T1521 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/collection1/lib/README' to classloader
[junit4:junit4] 2> 14426 T1521 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 14476 T1521 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 14496 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14497 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11140_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11140"}
[junit4:junit4] 2> 14510 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14511 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14511 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14510 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14510 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14538 T1521 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 14639 T1521 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 14647 T1521 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15289 T1521 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15299 T1521 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15302 T1521 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15313 T1521 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15317 T1521 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15321 T1521 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15322 T1521 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15322 T1521 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15322 T1521 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15323 T1521 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15323 T1521 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15324 T1521 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362937518682/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/
[junit4:junit4] 2> 15324 T1521 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ad6dbe
[junit4:junit4] 2> 15324 T1521 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15325 T1521 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3 forceNew: false
[junit4:junit4] 2> 15325 T1521 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index/
[junit4:junit4] 2> 15326 T1521 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15326 T1521 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index forceNew: false
[junit4:junit4] 2> 15328 T1521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15328 T1521 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15330 T1521 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15331 T1521 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15331 T1521 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15332 T1521 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15332 T1521 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15332 T1521 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15332 T1521 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15333 T1521 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15333 T1521 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15344 T1521 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15349 T1521 oass.SolrIndexSearcher.<init> Opening Searcher@7abf7d6b main
[junit4:junit4] 2> 15349 T1521 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/tlog
[junit4:junit4] 2> 15350 T1521 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15350 T1521 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15353 T1522 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7abf7d6b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15355 T1521 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15355 T1521 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16016 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16017 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11146_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11146"}
[junit4:junit4] 2> 16018 T1469 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 16018 T1469 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16030 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16030 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16030 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16030 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16030 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16392 T1521 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16392 T1521 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11146 collection:collection1 shard:shard1
[junit4:junit4] 2> 16395 T1521 oasc.ZkController.register We are http://127.0.0.1:11146/collection1/ and leader is http://127.0.0.1:11133/collection1/
[junit4:junit4] 2> 16395 T1521 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11146
[junit4:junit4] 2> 16395 T1521 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16395 T1521 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C74 name=collection1 org.apache.solr.core.SolrCore@10c32432 url=http://127.0.0.1:11146/collection1 node=127.0.0.1:11146_ C74_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:11146_, base_url=http://127.0.0.1:11146}
[junit4:junit4] 2> 16396 T1523 C74 P11146 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16396 T1521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16397 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16397 T1523 C74 P11146 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16397 T1523 C74 P11146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16397 T1446 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 16398 T1446 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16398 T1523 C74 P11146 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16398 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16412 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16413 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16421 T1479 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/onenodecollectioncore
[junit4:junit4] 2> 16422 T1479 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16422 T1479 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16423 T1479 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16424 T1479 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16424 T1479 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16426 T1479 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16428 T1479 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/onenodecollectioncore/'
[junit4:junit4] 2> 16498 T1479 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16560 T1479 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16661 T1479 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16669 T1479 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17296 T1479 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17305 T1479 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17308 T1479 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17321 T1479 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17326 T1479 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17329 T1479 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17331 T1479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17332 T1479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17332 T1479 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17333 T1479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17333 T1479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17333 T1479 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362937509566/onenodecollectioncore/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection/
[junit4:junit4] 2> 17334 T1479 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@27ad6dbe
[junit4:junit4] 2> 17334 T1479 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17335 T1479 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection forceNew: false
[junit4:junit4] 2> 17335 T1479 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection/index/
[junit4:junit4] 2> 17336 T1479 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17336 T1479 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection/index forceNew: false
[junit4:junit4] 2> 17338 T1479 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f225db3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@204f2b04),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17339 T1479 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17341 T1479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17341 T1479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17342 T1479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17342 T1479 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17343 T1479 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17343 T1479 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17343 T1479 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17344 T1479 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17344 T1479 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17354 T1479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17359 T1479 oass.SolrIndexSearcher.<init> Opening Searcher@6cfff1d3 main
[junit4:junit4] 2> 17360 T1479 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection/tlog
[junit4:junit4] 2> 17361 T1479 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17361 T1479 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17364 T1525 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@6cfff1d3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17365 T1479 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 17540 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17541 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11146_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11146"}
[junit4:junit4] 2> 17636 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:11133_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11133"}
[junit4:junit4] 2> 17637 T1469 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 17637 T1469 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17649 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17649 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17649 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17649 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17649 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18395 T1479 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 18396 T1479 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:11133 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 18397 T1479 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 18416 T1478 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=127.0.0.1:11146__collection1&state=recovering&nodeName=127.0.0.1:11146_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 18441 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:delete cxid:0xa1 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 18457 T1479 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 18458 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0xa2 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18467 T1479 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 18468 T1479 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 18468 T1479 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11133/onenodecollectioncore/
[junit4:junit4] 2> 18468 T1479 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 18469 T1479 oasc.SyncStrategy.syncToMe http://127.0.0.1:11133/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 18469 T1479 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11133/onenodecollectioncore/
[junit4:junit4] 2> 18469 T1479 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 18479 T1451 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d5567e7870006 type:create cxid:0xac zxid:0xb2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 19163 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19185 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19185 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19185 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19185 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19185 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19195 T1479 oasc.ZkController.register We are http://127.0.0.1:11133/onenodecollectioncore/ and leader is http://127.0.0.1:11133/onenodecollectioncore/
[junit4:junit4] 2> 19195 T1479 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:11133
[junit4:junit4] 2> 19196 T1479 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 19196 T1479 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 19198 T1479 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19200 T1479 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&roles=none&name=onenodecollectioncore&action=CREATE&collection=onenodecollection&dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362937505664/onenodecollection&wt=javabin&version=2} status=0 QTime=2779
[junit4:junit4] 2> 19200 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 19202 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20204 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C74_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:11146_, base_url=http://127.0.0.1:11146}
[junit4:junit4] 2> 20418 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:11133/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20418 T1523 C74 P11146 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:11146 START replicas=[http://127.0.0.1:11133/collection1/] nUpdates=100
[junit4:junit4] 2> 20418 T1523 C74 P11146 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20419 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20419 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20419 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20419 T1523 C74 P11146 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20420 T1523 C74 P11146 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:11133/collection1/. core=collection1
[junit4:junit4] 2> 20420 T1523 C74 P11146 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C75 name=collection1 org.apache.solr.core.SolrCore@18ec38ac url=http://127.0.0.1:11133/collection1 node=127.0.0.1:11133_ C75_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:11133_, base_url=http://127.0.0.1:11133, leader=true}
[junit4:junit4] 2> 20427 T1480 C75 P11133 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 20438 T1481 C75 P11133 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20441 T1481 C75 P11133 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20441 T1481 C75 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20442 T1481 C75 P11133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20442 T1481 C75 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20443 T1481 C75 P11133 oass.SolrIndexSearcher.<init> Opening Searcher@4663d467 realtime
[junit4:junit4] 2> 20443 T1481 C75 P11133 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20444 T1481 C75 P11133 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 20445 T1523 C74 P11146 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20445 T1523 C74 P11146 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20447 T1482 C75 P11133 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20447 T1482 C75 P11133 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 20448 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 20448 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20449 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20451 T1482 C75 P11133 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 20452 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20453 T1523 C74 P11146 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index.20130311044526111 forceNew: false
[junit4:junit4] 2> 20454 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b54e2d lockFactory=org.apache.lucene.store.NativeFSLockFactory@243be98a) fullCopy=false
[junit4:junit4] 2> 20456 T1482 C75 P11133 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 20457 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20459 T1523 C74 P11146 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20459 T1523 C74 P11146 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20461 T1523 C74 P11146 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20462 T1523 C74 P11146 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20462 T1523 C74 P11146 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20462 T1523 C74 P11146 oass.SolrIndexSearcher.<init> Opening Searcher@54207b48 main
[junit4:junit4] 2> 20463 T1522 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54207b48 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 20464 T1523 C74 P11146 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b54e2d lockFactory=org.apache.lucene.store.NativeFSLockFactory@243be98a)
[junit4:junit4] 2> 20464 T1523 C74 P11146 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20464 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20465 T1523 C74 P11146 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20465 T1523 C74 P11146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20466 T1523 C74 P11146 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20692 T1469 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20693 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:11133_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11133"}
[junit4:junit4] 2> 20697 T1469 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:11146_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11146"}
[junit4:junit4] 2> 20709 T1468 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20709 T1475 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20709 T1488 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20709 T1504 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20709 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21207 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21209 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 21209 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C76 name=onenodecollectioncore org.apache.solr.core.SolrCore@632ce7d5 url=http://127.0.0.1:11133/onenodecollectioncore node=127.0.0.1:11133_ C76_STATE=coll:onenodecollection core:onenodecollectioncore props:{shard=shard1, roles=none, state=active, core=onenodecollectioncore, collection=onenodecollection, node_name=127.0.0.1:11133_, base_url=http://127.0.0.1:11133, leader=true}
[junit4:junit4] 2> 21230 T1483 C76 P11133 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f225db3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@204f2b04),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21231 T1483 C76 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21234 T1483 C76 P11133 /update {wt=javabin&version=2} {add=[1 (1429143580193062912)]} 0 6
[junit4:junit4] 2> 21238 T1483 C76 P11133 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21241 T1483 C76 P11133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f225db3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@204f2b04),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f225db3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@204f2b04),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Asserting_0.dvd, _0_MockSep_0.pos, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0.fdt]
[junit4:junit4] 2> 21242 T1483 C76 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Asserting_0.dvd, _0_MockSep_0.pos, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0.si, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0.fdt]
[junit4:junit4] 2> 21243 T1483 C76 P11133 oass.SolrIndexSearcher.<init> Opening Searcher@5101b1fb main
[junit4:junit4] 2> 21244 T1483 C76 P11133 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21245 T1525 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@5101b1fb main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21245 T1483 C76 P11133 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 21250 T1483 C76 P11133 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21251 T1446 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 21264 T1483 C76 P11133 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C77 name=collection1 org.apache.solr.core.SolrCore@1297c4de url=http://127.0.0.1:11127/collection1 node=127.0.0.1:11127_ C77_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:11127_, base_url=http://127.0.0.1:11127, leader=true}
[junit4:junit4] 2> 21276 T1458 C77 P11127 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21276 T1458 C77 P11127 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21281 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[1 (1429143580240248832)]} 0 8
[junit4:junit4] 2> C74_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:11146_, base_url=http://127.0.0.1:11146}
[junit4:junit4] 2> 21308 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429143580259123200)]} 0 5
[junit4:junit4] 2> 21309 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[1 (1429143580259123200)]} 0 17
[junit4:junit4] 2> 21312 T1458 C77 P11127 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21318 T1458 C77 P11127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Asserting_0.dvd, _0_MockSep_0.pos, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_MockRandom_0.doc, _0_Pulsing41_0.tip, _0_MockRandom_0.pos, _0.nvd, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0.si, _0_MockRandom_0.tim, _0.fdt]
[junit4:junit4] 2> 21319 T1458 C77 P11127 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Asserting_0.dvd, _0_MockSep_0.pos, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_MockRandom_0.doc, _0_Pulsing41_0.tip, _0_MockRandom_0.pos, _0.nvd, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0.si, _0_MockRandom_0.tim, _0.fdt]
[junit4:junit4] 2> 21322 T1458 C77 P11127 oass.SolrIndexSearcher.<init> Opening Searcher@283c035 main
[junit4:junit4] 2> 21322 T1458 C77 P11127 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21323 T1472 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@283c035 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21323 T1458 C77 P11127 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 21326 T1484 C75 P11133 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21356 T1484 C75 P11133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockRandom_0.skp, _0_MockSep_0.tib, _0_MockRandom_0.frq, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> 21357 T1484 C75 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockSep_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockRandom_0.skp, _0_MockSep_0.tib, _0_MockRandom_0.frq, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> 21378 T1484 C75 P11133 oass.SolrIndexSearcher.<init> Opening Searcher@183a21a0 main
[junit4:junit4] 2> 21379 T1484 C75 P11133 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21380 T1490 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@183a21a0 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21380 T1484 C75 P11133 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:11146/collection1/, StdNode: http://127.0.0.1:11140/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 21384 T1510 C74 P11146 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C78 name=collection1 org.apache.solr.core.SolrCore@a3d5b42 url=http://127.0.0.1:11140/collection1 node=127.0.0.1:11140_ C78_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:11140_, base_url=http://127.0.0.1:11140, leader=true}
[junit4:junit4] 2> 21390 T1496 C78 P11140 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21390 T1510 C74 P11146 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.sd, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> 21391 T1510 C74 P11146 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockSep_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.sd, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> 21392 T1496 C78 P11140 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21393 T1496 C78 P11140 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21394 T1496 C78 P11140 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 21394 T1496 C78 P11140 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 21395 T1496 C78 P11140 oass.SolrIndexSearcher.<init> Opening Searcher@5cd1f5b5 main
[junit4:junit4] 2> 21395 T1496 C78 P11140 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21396 T1506 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5cd1f5b5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 21397 T1496 C78 P11140 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 21444 T1510 C74 P11146 oass.SolrIndexSearcher.<init> Opening Searcher@39471ba9 main
[junit4:junit4] 2> 21445 T1510 C74 P11146 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21445 T1522 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39471ba9 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21446 T1510 C74 P11146 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 63
[junit4:junit4] 2> 21447 T1484 C75 P11133 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 121
[junit4:junit4] 2> 21447 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21448 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21449 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 21453 T1458 C77 P11127 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21454 T1446 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11124/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b69119f
[junit4:junit4] 2> 21455 T1446 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 21455 T1531 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11124. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 21456 T1531 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11124, initiating session
[junit4:junit4] 2> 21456 T1448 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11185
[junit4:junit4] 2> 21456 T1448 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11185
[junit4:junit4] 2> 21458 T1450 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d5567e787000b with negotiated timeout 10000 for client /140.211.11.196:11185
[junit4:junit4] 2> 21458 T1531 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11124, sessionid = 0x13d5567e787000b, negotiated timeout = 10000
[junit4:junit4] 2> 21458 T1532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b69119f name:ZooKeeperConnection Watcher:127.0.0.1:11124/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 21458 T1446 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 21460 T1446 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 21462 T1451 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d5567e787000b
[junit4:junit4] 2> 21463 T1532 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21463 T1448 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:11185 which had sessionid 0x13d5567e787000b
[junit4:junit4] 2> 21463 T1446 oaz.ZooKeeper.close Session: 0x13d5567e787000b closed
[junit4:junit4] 2> 21466 T1478 C75 P11133 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21468 T1497 C78 P11140 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21473 T1499 C78 P11140 REQ /select {NOW=1362937527128&shard.url=127.0.0.1:11140/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21473 T1511 C74 P11146 REQ /select {NOW=1362937527128&shard.url=127.0.0.1:11133/collection1/|127.0.0.1:11146/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21477 T1511 C74 P11146 REQ /select {NOW=1362937527128&shard.url=127.0.0.1:11133/collection1/|127.0.0.1:11146/collection1/&ids=1&q=*:*&distrib=false&wt=javabin&isShard=true&version=2} status=0 QTime=0
[junit4:junit4] 2> 21478 T1498 C78 P11140 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=8
[junit4:junit4] 2> 21488 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[2 (1429143580461498368)]} 0 4
[junit4:junit4] 2> 21498 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1429143580471984128)]} 0 4
[junit4:junit4] 2> 21499 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[2]} 0 8
[junit4:junit4] 2> 21504 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[3 (1429143580480372736)]} 0 2
[junit4:junit4] 2> 21510 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[3 (1429143580485615616)]} 0 3
[junit4:junit4] 2> 21516 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[4 (1429143580491907072)]} 0 3
[junit4:junit4] 2> 21529 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1429143580498198528)]} 0 4
[junit4:junit4] 2> 21529 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[4 (1429143580498198528)]} 0 10
[junit4:junit4] 2> 21535 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[5 (1429143580512878592)]} 0 2
[junit4:junit4] 2> 21541 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[5 (1429143580518121472)]} 0 3
[junit4:junit4] 2> 21546 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[6 (1429143580524412928)]} 0 2
[junit4:junit4] 2> 21555 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1429143580532801536)]} 0 3
[junit4:junit4] 2> 21555 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[6]} 0 6
[junit4:junit4] 2> 21561 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[7 (1429143580539092992)]} 0 3
[junit4:junit4] 2> 21567 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[7 (1429143580545384448)]} 0 2
[junit4:junit4] 2> 21573 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[8 (1429143580551675904)]} 0 3
[junit4:junit4] 2> 21584 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1429143580557967360)]} 0 3
[junit4:junit4] 2> 21584 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[8 (1429143580557967360)]} 0 8
[junit4:junit4] 2> 21590 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[9 (1429143580569501696)]} 0 3
[junit4:junit4] 2> 21596 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[9 (1429143580575793152)]} 0 3
[junit4:junit4] 2> 21601 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[10 (1429143580582084608)]} 0 2
[junit4:junit4] 2> 21611 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1429143580587327488)]} 0 2
[junit4:junit4] 2> 21612 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[10 (1429143580587327488)]} 0 8
[junit4:junit4] 2> 21617 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[11 (1429143580598861824)]} 0 2
[junit4:junit4] 2> 21628 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1429143580604104704)]} 0 3
[junit4:junit4] 2> 21628 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[11 (1429143580604104704)]} 0 8
[junit4:junit4] 2> 21633 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[12 (1429143580615639040)]} 0 2
[junit4:junit4] 2> 21644 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1429143580620881920)]} 0 3
[junit4:junit4] 2> 21645 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[12 (1429143580620881920)]} 0 9
[junit4:junit4] 2> 21651 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[13 (1429143580633464832)]} 0 3
[junit4:junit4] 2> 21668 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1429143580642902016)]} 0 4
[junit4:junit4] 2> 21669 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1429143580642902016)]} 0 11
[junit4:junit4] 2> 21669 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[13]} 0 15
[junit4:junit4] 2> 21676 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[14 (1429143580659679232)]} 0 3
[junit4:junit4] 2> 21686 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1429143580665970688)]} 0 2
[junit4:junit4] 2> 21687 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[14 (1429143580665970688)]} 0 8
[junit4:junit4] 2> 21692 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[15 (1429143580677505024)]} 0 2
[junit4:junit4] 2> 21702 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1429143580682747904)]} 0 2
[junit4:junit4] 2> 21703 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[15 (1429143580682747904)]} 0 8
[junit4:junit4] 2> 21708 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[16 (1429143580694282240)]} 0 2
[junit4:junit4] 2> 21719 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1429143580700573696)]} 0 2
[junit4:junit4] 2> 21720 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[16 (1429143580700573696)]} 0 8
[junit4:junit4] 2> 21731 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[17 (1429143580715253760)]} 0 7
[junit4:junit4] 2> 21747 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[17 (1429143580732030976)]} 0 7
[junit4:junit4] 2> 21748 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[17]} 0 14
[junit4:junit4] 2> 21753 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[100 (1429143580741468160)]} 0 2
[junit4:junit4] 2> 21766 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[100 (1429143580749856768)]} 0 2
[junit4:junit4] 2> 21767 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[100 (1429143580749856768)]} 0 8
[junit4:junit4] 2> 21768 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[100]} 0 12
[junit4:junit4] 2> 21773 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[101 (1429143580762439680)]} 0 2
[junit4:junit4] 2> 21787 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[101 (1429143580771876864)]} 0 2
[junit4:junit4] 2> 21789 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[101 (1429143580771876864)]} 0 9
[junit4:junit4] 2> 21789 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[101]} 0 12
[junit4:junit4] 2> 21795 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[102 (1429143580784459776)]} 0 3
[junit4:junit4] 2> 21805 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[102 (1429143580790751232)]} 0 2
[junit4:junit4] 2> 21806 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[102 (1429143580790751232)]} 0 8
[junit4:junit4] 2> 21813 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[103 (1429143580803334144)]} 0 3
[junit4:junit4] 2> 21819 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[103 (1429143580809625600)]} 0 3
[junit4:junit4] 2> 21824 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[104 (1429143580815917056)]} 0 2
[junit4:junit4] 2> 21831 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[104 (1429143580822208512)]} 0 3
[junit4:junit4] 2> 21836 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[105 (1429143580828499968)]} 0 2
[junit4:junit4] 2> 21844 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[105 (1429143580836888576)]} 0 2
[junit4:junit4] 2> 21846 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[105]} 0 7
[junit4:junit4] 2> 21851 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[106 (1429143580844228608)]} 0 2
[junit4:junit4] 2> 21859 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[106 (1429143580852617216)]} 0 2
[junit4:junit4] 2> 21860 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[106]} 0 6
[junit4:junit4] 2> 21865 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[107 (1429143580858908672)]} 0 2
[junit4:junit4] 2> 21876 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[107 (1429143580864151552)]} 0 3
[junit4:junit4] 2> 21877 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[107 (1429143580864151552)]} 0 9
[junit4:junit4] 2> 21882 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[108 (1429143580876734464)]} 0 2
[junit4:junit4] 2> 21893 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[108 (1429143580881977344)]} 0 2
[junit4:junit4] 2> 21893 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[108 (1429143580881977344)]} 0 8
[junit4:junit4] 2> 21899 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[109 (1429143580894560256)]} 0 2
[junit4:junit4] 2> 21908 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[109 (1429143580899803136)]} 0 2
[junit4:junit4] 2> 21909 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[109 (1429143580899803136)]} 0 7
[junit4:junit4] 2> 21915 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[110 (1429143580911337472)]} 0 2
[junit4:junit4] 2> 21926 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[110 (1429143580916580352)]} 0 3
[junit4:junit4] 2> 21927 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[110 (1429143580916580352)]} 0 9
[junit4:junit4] 2> 21933 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[111 (1429143580929163264)]} 0 3
[junit4:junit4] 2> 21943 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[111 (1429143580935454720)]} 0 3
[junit4:junit4] 2> 21944 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[111 (1429143580935454720)]} 0 8
[junit4:junit4] 2> 21949 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[112 (1429143580946989056)]} 0 2
[junit4:junit4] 2> 21954 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[112 (1429143580952231936)]} 0 2
[junit4:junit4] 2> 21959 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[113 (1429143580957474816)]} 0 2
[junit4:junit4] 2> 21972 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[113 (1429143580965863424)]} 0 2
[junit4:junit4] 2> 21973 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[113 (1429143580965863424)]} 0 8
[junit4:junit4] 2> 21974 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[113]} 0 12
[junit4:junit4] 2> 21980 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[114 (1429143580978446336)]} 0 3
[junit4:junit4] 2> 21991 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[114 (1429143580984737792)]} 0 2
[junit4:junit4] 2> 21991 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[114 (1429143580984737792)]} 0 8
[junit4:junit4] 2> 21998 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[115 (1429143580997320704)]} 0 3
[junit4:junit4] 2> 22003 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[115 (1429143581003612160)]} 0 2
[junit4:junit4] 2> 22010 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[116 (1429143581009903616)]} 0 2
[junit4:junit4] 2> 22018 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[116 (1429143581019340800)]} 0 2
[junit4:junit4] 2> 22019 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[116]} 0 6
[junit4:junit4] 2> 22024 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[117 (1429143581025632256)]} 0 2
[junit4:junit4] 2> 22029 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[117 (1429143581030875136)]} 0 2
[junit4:junit4] 2> 22034 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[118 (1429143581036118016)]} 0 2
[junit4:junit4] 2> 22046 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[118 (1429143581044506624)]} 0 2
[junit4:junit4] 2> 22047 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[118 (1429143581044506624)]} 0 7
[junit4:junit4] 2> 22048 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[118]} 0 11
[junit4:junit4] 2> 22053 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[119 (1429143581056040960)]} 0 2
[junit4:junit4] 2> 22059 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[119 (1429143581062332416)]} 0 2
[junit4:junit4] 2> 22065 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[120 (1429143581068623872)]} 0 2
[junit4:junit4] 2> 22073 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[120 (1429143581077012480)]} 0 2
[junit4:junit4] 2> 22074 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[120]} 0 6
[junit4:junit4] 2> 22079 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[121 (1429143581083303936)]} 0 2
[junit4:junit4] 2> 22087 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[121 (1429143581091692544)]} 0 2
[junit4:junit4] 2> 22088 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[121]} 0 6
[junit4:junit4] 2> 22093 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[122 (1429143581097984000)]} 0 2
[junit4:junit4] 2> 22106 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[122 (1429143581106372608)]} 0 2
[junit4:junit4] 2> 22107 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[122 (1429143581106372608)]} 0 8
[junit4:junit4] 2> 22108 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[122]} 0 12
[junit4:junit4] 2> 22113 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[123 (1429143581118955520)]} 0 2
[junit4:junit4] 2> 22122 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[123 (1429143581124198400)]} 0 2
[junit4:junit4] 2> 22123 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[123 (1429143581124198400)]} 0 7
[junit4:junit4] 2> 22128 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[124 (1429143581134684160)]} 0 2
[junit4:junit4] 2> 22141 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[124 (1429143581143072768)]} 0 2
[junit4:junit4] 2> 22142 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[124 (1429143581143072768)]} 0 8
[junit4:junit4] 2> 22143 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[124]} 0 12
[junit4:junit4] 2> 22148 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[125 (1429143581155655680)]} 0 2
[junit4:junit4] 2> 22158 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[125 (1429143581160898560)]} 0 3
[junit4:junit4] 2> 22158 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[125 (1429143581160898560)]} 0 7
[junit4:junit4] 2> 22165 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[126 (1429143581172432896)]} 0 3
[junit4:junit4] 2> 22170 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[126 (1429143581177675776)]} 0 2
[junit4:junit4] 2> 22175 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[127 (1429143581182918656)]} 0 2
[junit4:junit4] 2> 22178 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[127 (1429143581187112960)]} 0 1
[junit4:junit4] 2> 22183 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[128 (1429143581192355840)]} 0 1
[junit4:junit4] 2> 22190 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[128 (1429143581199695872)]} 0 1
[junit4:junit4] 2> 22191 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[128]} 0 5
[junit4:junit4] 2> 22196 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[129 (1429143581205987328)]} 0 2
[junit4:junit4] 2> 22209 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[129 (1429143581214375936)]} 0 2
[junit4:junit4] 2> 22210 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[129 (1429143581214375936)]} 0 7
[junit4:junit4] 2> 22211 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[129]} 0 11
[junit4:junit4] 2> 22215 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[130 (1429143581225910272)]} 0 1
[junit4:junit4] 2> 22221 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[130 (1429143581232201728)]} 0 1
[junit4:junit4] 2> 22222 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[130]} 0 4
[junit4:junit4] 2> 22227 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[131 (1429143581238493184)]} 0 2
[junit4:junit4] 2> 22236 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[131 (1429143581246881792)]} 0 2
[junit4:junit4] 2> 22237 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[131]} 0 6
[junit4:junit4] 2> 22242 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[132 (1429143581253173248)]} 0 2
[junit4:junit4] 2> 22254 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[132 (1429143581262610432)]} 0 2
[junit4:junit4] 2> 22255 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[132 (1429143581262610432)]} 0 7
[junit4:junit4] 2> 22256 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[132]} 0 11
[junit4:junit4] 2> 22261 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[133 (1429143581273096192)]} 0 2
[junit4:junit4] 2> 22272 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[133 (1429143581280436224)]} 0 2
[junit4:junit4] 2> 22273 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[133 (1429143581280436224)]} 0 7
[junit4:junit4] 2> 22274 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[133]} 0 10
[junit4:junit4] 2> 22279 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[134 (1429143581291970560)]} 0 2
[junit4:junit4] 2> 22287 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[134 (1429143581301407744)]} 0 2
[junit4:junit4] 2> 22288 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[134]} 0 6
[junit4:junit4] 2> 22292 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[135 (1429143581306650624)]} 0 1
[junit4:junit4] 2> 22304 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[135 (1429143581315039232)]} 0 2
[junit4:junit4] 2> 22304 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[135 (1429143581315039232)]} 0 6
[junit4:junit4] 2> 22305 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[135]} 0 10
[junit4:junit4] 2> 22309 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[136 (1429143581324476416)]} 0 1
[junit4:junit4] 2> 22313 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[136 (1429143581328670720)]} 0 1
[junit4:junit4] 2> 22317 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[137 (1429143581332865024)]} 0 1
[junit4:junit4] 2> 22323 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[137 (1429143581339156480)]} 0 1
[junit4:junit4] 2> 22324 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[137]} 0 5
[junit4:junit4] 2> 22327 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[138 (1429143581343350784)]} 0 1
[junit4:junit4] 2> 22338 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[138 (1429143581350690816)]} 0 1
[junit4:junit4] 2> 22339 T1481 C75 P11133 /update {distrib.from=http://127.0.0.1:11140/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[138 (1429143581350690816)]} 0 6
[junit4:junit4] 2> 22340 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[138]} 0 10
[junit4:junit4] 2> 22344 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[139 (1429143581361176576)]} 0 1
[junit4:junit4] 2> 22348 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[139 (1429143581365370880)]} 0 1
[junit4:junit4] 2> 22352 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[140 (1429143581369565184)]} 0 1
[junit4:junit4] 2> 22356 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[140 (1429143581373759488)]} 0 1
[junit4:junit4] 2> 22360 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[141 (1429143581377953792)]} 0 1
[junit4:junit4] 2> 22364 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[141 (1429143581382148096)]} 0 1
[junit4:junit4] 2> 22368 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[142 (1429143581386342400)]} 0 1
[junit4:junit4] 2> 22372 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[142 (1429143581390536704)]} 0 1
[junit4:junit4] 2> 22376 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[143 (1429143581394731008)]} 0 1
[junit4:junit4] 2> 22384 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[143 (1429143581402071040)]} 0 2
[junit4:junit4] 2> 22384 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[143]} 0 5
[junit4:junit4] 2> 22388 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[144 (1429143581407313920)]} 0 1
[junit4:junit4] 2> 22392 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[144 (1429143581411508224)]} 0 1
[junit4:junit4] 2> 22396 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[145 (1429143581415702528)]} 0 1
[junit4:junit4] 2> 22402 T1496 C78 P11140 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[145 (1429143581421993984)]} 0 1
[junit4:junit4] 2> 22402 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[145]} 0 4
[junit4:junit4] 2> 22406 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[146 (1429143581426188288)]} 0 1
[junit4:junit4] 2> 22414 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[146 (1429143581430382592)]} 0 2
[junit4:junit4] 2> 22414 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[146 (1429143581430382592)]} 0 5
[junit4:junit4] 2> 22418 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[147 (1429143581438771200)]} 0 1
[junit4:junit4] 2> 22422 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[147 (1429143581442965504)]} 0 1
[junit4:junit4] 2> 22426 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[148 (1429143581447159808)]} 0 1
[junit4:junit4] 2> 22435 T1510 C74 P11146 /update {distrib.from=http://127.0.0.1:11133/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[148 (1429143581451354112)]} 0 2
[junit4:junit4] 2> 22435 T1484 C75 P11133 /update {wt=javabin&version=2} {add=[148 (1429143581451354112)]} 0 6
[junit4:junit4] 2> 22439 T1458 C77 P11127 /update {wt=javabin&version=2} {add=[149 (1429143581460791296)]} 0 1
[junit4:junit4] 2> 22443 T1498 C78 P11140 /update {wt=javabin&version=2} {add=[149 (1429143581464985600)]} 0 1
[junit4:junit4] 2> 22443 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 22445 T1446 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22446 T1446 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 22448 T1458 C77 P11127 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22476 T1458 C77 P11127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Asserting_0.dvd, _0_MockSep_0.pos, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_MockRandom_0.doc, _0_Pulsing41_0.tip, _0_MockRandom_0.pos, _0.nvd, _0_MockSep_0.frq, segments_2, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockSep_0.doc, _0.si, _0_MockRandom_0.tim, _0.fdt]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f328413 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c772b37),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _1.fnm, _1_MockRandom_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _1_MockSep_0.tib, _1_MockRandom_0.tip, _1.nvd, _1_MockRandom_0.tim, _1_MockRandom_0.sd, _1_Pulsing41_0.doc, _0_Asserting_0.dvm, _1_MockSep_0.tii, _1.fdx, _1_Pulsing41_0.tip, _1.fdt, _0_MockRandom_0.doc, _1_Pulsing41_0.tim, _0_MockRandom_0.pos, _1_MockRandom_0.doc, _1_MockSep_0.frq, _1_Memory_0.ram, _0.nvd, _0_MockSep_0.doc, _0.si, _1_MockSep_0.pos, _0.nvm, _1_Pulsing41_0.pos, _0_MockSep_0.tib, _1_Asserting_0.dvm, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _1.si, _1_Asserting_0.dvd, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.tim, segments_3, _0.fdt]
[junit4:junit4] 2> 22477 T1458 C77 P11127 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockSep_0.pos, _1.fnm, _1_MockRandom_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _1_MockSep_0.tib, _1_MockRandom_0.tip, _1.nvd, _1_MockRandom_0.tim, _1_MockRandom_0.sd, _1_Pulsing41_0.doc, _0_Asserting_0.dvm, _1_MockSep_0.tii, _1.fdx, _1_Pulsing41_0.tip, _1.fdt, _0_MockRandom_0.doc, _1_Pulsing41_0.tim, _0_MockRandom_0.pos, _1_MockRandom_0.doc, _1_MockSep_0.frq, _1_Memory_0.ram, _0.nvd, _0_MockSep_0.doc, _0.si, _1_MockSep_0.pos, _0.nvm, _1_Pulsing41_0.pos, _0_MockSep_0.tib, _1_Asserting_0.dvm, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tip, _1.si, _1_Asserting_0.dvd, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.tim, segments_3, _0.fdt]
[junit4:junit4] 2> 22479 T1458 C77 P11127 oass.SolrIndexSearcher.<init> Opening Searcher@434a66bb main
[junit4:junit4] 2> 22479 T1458 C77 P11127 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22480 T1472 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@434a66bb main{StandardDirectoryReader(segments_3:5 _0(5.0):C1 _1(5.0):C66)}
[junit4:junit4] 2> 22480 T1458 C77 P11127 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> 22483 T1498 C78 P11140 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22501 T1498 C78 P11140 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6f77e7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2613695e),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_MockRandom_0.tip, _0_MockRandom_0.sd, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 22502 T1498 C78 P11140 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockSep_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_MockRandom_0.tip, _0_MockRandom_0.sd, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.tim, _0.fdt, segments_3]
[junit4:junit4] 2> 22504 T1498 C78 P11140 oass.SolrIndexSearcher.<init> Opening Searcher@2208de1 main
[junit4:junit4] 2> 22504 T1498 C78 P11140 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22505 T1506 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2208de1 main{StandardDirectoryReader(segments_3:3 _0(5.0):C35)}
[junit4:junit4] 2> 22505 T1498 C78 P11140 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:11133/collection1/, StdNode: http://127.0.0.1:11146/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 22508 T1512 C74 P11146 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22508 T1481 C75 P11133 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22539 T1512 C74 P11146 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockSep_0.tib, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.sd, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@40784d5a lockFactory=org.apache.lucene.store.NativeFSLockFactory@447e2636),segFN=segments_4,generation=4,filenames=[_0_MockSep_0.pos, _1.fnm, _1_MockRandom_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _1_MockSep_0.tib, _1_MockRandom_0.tip, _1.nvd, _1_MockRandom_0.tim, _1_MockRandom_0.sd, _1_Pulsing41_0.doc, _0_Asserting_0.dvm, _1_MockSep_0.tii, _1.fdx, _1_Pulsing41_0.tip, _1.fdt, _0_MockRandom_0.doc, _1_Pulsing41_0.tim, _0_MockRandom_0.pos, _1_MockRandom_0.doc, _1_MockSep_0.frq, _1_Memory_0.ram, _0.nvd, _0_MockSep_0.doc, _0.si, _0_MockRandom_0.tib, _1_MockSep_0.pos, _0.nvm, _1_Pulsing41_0.pos, _0_MockSep_0.tib, _1_Asserting_0.dvm, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tiv, _1.si, _1_Asserting_0.dvd, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0.fdt, segments_4]
[junit4:junit4] 2> 22539 T1481 C75 P11133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_3,generation=3,filenames=[_0_MockSep_0.pos, _0_MockSep_0.skp, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.si, _0_MockSep_0.doc, _0_MockRandom_0.tib, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _0_Asserting_0.dvm, _0.nvm, _0_MockRandom_0.skp, _0_MockSep_0.tib, _0_MockRandom_0.frq, _0_MockRandom_0.sd, _0_Pulsing41_0.pos, _0_Pulsing41_0.doc, _0_MockRandom_0.tiv, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockSep_0.pyl, _0_MockRandom_0.pyl, _0.fdt, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aaae79c lockFactory=org.apache.lucene.store.NativeFSLockFactory@334760d2),segFN=segments_4,generation=4,filenames=[_0_MockSep_0.pos, _1.fnm, _1_MockRandom_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _1_MockSep_0.tib, _1_MockRandom_0.tip, _1.nvd, _1_MockRandom_0.tim, _1_MockRandom_0.sd, _1_Pulsing41_0.doc, _0_Asserting_0.dvm, _0_MockRandom_0.skp, _1_MockSep_0.tii, _1.fdx, _0_MockRandom_0.frq, _1_Pulsing41_0.tip, _1.fdt, _1_Pulsing41_0.tim, _0_MockRandom_0.doc, _0_MockRandom_0.pos, _1_MockRandom_0.doc, _1_MockSep_0.frq, _1_Memory_0.ram, _0.nvd, _0_MockSep_0.doc, _0.si, _0_MockRandom_0.tib, _1_MockSep_0.pos, _0.nvm, _1_Pulsing41_0.pos, _0_MockSep_0.tib, _1_Asserting_0.dvm, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_MockRandom_0.sd, _0_MockRandom_0.tiv, _1.si, _1_Asserting_0.dvd, _0_MockSep_0.frq, _0.fdx, _0_MockSep_0.tii, _0_MockRandom_0.pyl, _0_MockSep_0.pyl, _0.fdt, segments_4]
[junit4:junit4] 2> 22541 T1481 C75 P11133 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_0_MockSep_0.pos, _1.fnm, _1_MockRandom_0.pos, _0_Pulsing41_0.tim, _0_MockSep_0.skp, _0_Pulsing41_0.tip, _1_MockSep_0.pyl, _1_MockSep_0.doc, _1.nvm, _1_MockSep_0.skp, _0.fnm, _0_Asserting_0.dvd, _0_Memory_0.ram, _1_MockSep_0.tib, _1_MockRandom_0.tip, _1.nvd, _1_MockRandom_0.tim, _1_MockRandom_0.sd, _1_Pulsing41_0.doc, _0_Asserting_0.dvm, _0_MockRandom_0.skp, _1_MockSep_0.tii, _1.fdx, _0_MockRandom_0.frq, _1_Pulsing41_0.tip, _1.f
[...truncated too long message...]
ctoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3
[junit4:junit4] 2> 87046 T1520 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3
[junit4:junit4] 2> 87047 T1520 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index.20130311044526111
[junit4:junit4] 2> 87047 T1520 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362937505664/jetty3/index.20130311044526111
[junit4:junit4] 2> 87047 T1520 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 87047 T1520 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/89321472371654666-127.0.0.1:11146__collection1-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:153)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:243)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 87048 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 87048 T1520 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87049 T1520 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87049 T1520 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 87049 T1520 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 87049 T1520 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 87050 T1520 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {foo_b=PostingsFormat(name=MockRandom), n_f1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), foo_d=PostingsFormat(name=MockSep), foo_f=PostingsFormat(name=MockRandom), n_tl1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), n_d1=PostingsFormat(name=Memory doPackFST= false), rnd_b=PostingsFormat(name=MockSep), intDefault=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), n_td1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), timestamp=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=MockSep), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=Memory doPackFST= false), oddField_s=PostingsFormat(name=MockSep), multiDefault=PostingsFormat(name=Memory doPackFST= false), n_tf1=PostingsFormat(name=Memory doPackFST= false), n_dt1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), range_facet_l=PostingsFormat(name=Memory doPackFST= false), n_ti1=PostingsFormat(name=MockSep), text=PostingsFormat(name=MockRandom), _version_=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), SubjectTerms_mfacet=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), a_t=PostingsFormat(name=Memory doPackFST= false), n_tdt1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), other_tl1=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136), n_l1=PostingsFormat(name=Memory doPackFST= false), a_si=Pulsing41(freqCutoff=12 minBlockSize=69 maxBlockSize=136)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=sr, timezone=Australia/Hobart
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=162904296,total=519045120
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestZkChroot, ShardRoutingCustomTest, TestBM25SimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestCSVLoader, TestDFRSimilarityFactory, IndexBasedSpellCheckerTest, TestFiltering, TestOmitPositions, IndexSchemaRuntimeFieldTest, TestAnalyzedSuggestions, HardAutoCommitTest, LeaderElectionIntegrationTest, DirectUpdateHandlerTest, AliasIntegrationTest, ZkNodePropsTest, TestBinaryField, TimeZoneUtilsTest, TestSuggestSpellingConverter, DateMathParserTest, TestJmxMonitoredMap, SignatureUpdateProcessorFactoryTest, TestSolrXMLSerializer, TestDynamicFieldCollectionResource, UpdateParamsTest, StatsComponentTest, OpenExchangeRatesOrgProviderTest, PingRequestHandlerTest, NotRequiredUniqueKeyTest, BadIndexSchemaTest, TestElisionMultitermQuery, SolrPluginUtilsTest, CoreContainerCoreInitFailuresTest, TestQueryTypes, EchoParamsTest, TestFieldTypeResource, ConvertedLegacyTest, ChaosMonkeySafeLeaderTest, JSONWriterTest, TestHashPartitioner, SpellCheckComponentTest, DisMaxRequestHandlerTest, TestComponentsName, SuggesterWFSTTest, URLClassifyProcessorTest, HighlighterTest, TestAtomicUpdateErrorCases, TestStressReorder, TestLazyCores, NoCacheHeaderTest, TestDynamicFieldResource, XmlUpdateRequestHandlerTest, TestCSVResponseWriter, TestIndexSearcher, TestIndexingPerformance, BasicZkTest, DistanceFunctionTest, TermVectorComponentDistributedTest, TestUtils, TestSolrJ, TestQuerySenderNoQuery, SpellCheckCollatorTest, TestSystemIdResolver, XsltUpdateRequestHandlerTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 87.85s, 1 test, 1 error <<< FAILURES!
[...truncated 553 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 269 suites, 1137 tests, 1 error, 27 ignored (8 assumptions)
Total time: 49 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3798 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3798/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.lucene.index.sorter.IndexSortingTest
Error Message:
37
Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: 37
at __randomizedtesting.SeedInfo.seed([ECF2C004E4BBAB07]:0)
at org.apache.lucene.index.sorter.SortingAtomicReader$SortingDocsAndPositionsEnum.<init>(SortingAtomicReader.java:454)
at org.apache.lucene.index.sorter.SortingAtomicReader$SortingTermsEnum.docsAndPositions(SortingAtomicReader.java:161)
at org.apache.lucene.index.MultiTermsEnum.docsAndPositions(MultiTermsEnum.java:474)
at org.apache.lucene.index.TermsEnum.docsAndPositions(TermsEnum.java:183)
at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:186)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:383)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:116)
at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2420)
at org.apache.lucene.index.sorter.IndexSortingTest.beforeClassSorterUtilTest(IndexSortingTest.java:65)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.sorter.SortingAtomicReaderTest
Error Message:
CheckReader failed
Stack Trace:
java.lang.RuntimeException: CheckReader failed
at __randomizedtesting.SeedInfo.seed([ECF2C004E4BBAB07]:0)
at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:255)
at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:234)
at org.apache.lucene.index.sorter.SortingAtomicReaderTest.beforeClassSortingAtomicReaderTest(SortingAtomicReaderTest.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 7360 lines...]
[junit4:junit4] Suite: org.apache.lucene.index.sorter.IndexSortingTest
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {id=Pulsing41(freqCutoff=7 minBlockSize=98 maxBlockSize=218), positions=PostingsFormat(name=Memory doPackFST= false), norm=PostingsFormat(name=Lucene41WithOrds), docs=Pulsing41(freqCutoff=7 minBlockSize=98 maxBlockSize=218), term_vectors=Pulsing41(freqCutoff=7 minBlockSize=98 maxBlockSize=218)}, docValues:{binary=DocValuesFormat(name=Disk), numeric=DocValuesFormat(name=Disk), sorted=DocValuesFormat(name=Disk), sorted_set=DocValuesFormat(name=Lucene42)}, sim=DefaultSimilarity, locale=bg, timezone=America/Regina
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=1,free=95742640,total=125829120
[junit4:junit4] 2> NOTE: All tests run in this JVM: [IndexSortingTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=IndexSortingTest -Dtests.seed=ECF2C004E4BBAB07 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=America/Regina -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | IndexSortingTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.ArrayIndexOutOfBoundsException: 37
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([ECF2C004E4BBAB07]:0)
[junit4:junit4] > at org.apache.lucene.index.sorter.SortingAtomicReader$SortingDocsAndPositionsEnum.<init>(SortingAtomicReader.java:454)
[junit4:junit4] > at org.apache.lucene.index.sorter.SortingAtomicReader$SortingTermsEnum.docsAndPositions(SortingAtomicReader.java:161)
[junit4:junit4] > at org.apache.lucene.index.MultiTermsEnum.docsAndPositions(MultiTermsEnum.java:474)
[junit4:junit4] > at org.apache.lucene.index.TermsEnum.docsAndPositions(TermsEnum.java:183)
[junit4:junit4] > at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:186)
[junit4:junit4] > at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
[junit4:junit4] > at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:383)
[junit4:junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:116)
[junit4:junit4] > at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:2420)
[junit4:junit4] > at org.apache.lucene.index.sorter.IndexSortingTest.beforeClassSorterUtilTest(IndexSortingTest.java:65)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] Completed on J0 in 2.93s, 0 tests, 1 error <<< FAILURES!
[...truncated 13 lines...]
[junit4:junit4] Suite: org.apache.lucene.index.sorter.SortingAtomicReaderTest
[junit4:junit4] 1> CheckReader failed
[junit4:junit4] 1> test: field norms.........OK [3 fields]
[junit4:junit4] 1> test: terms, freq, prox...ERROR: java.lang.ArrayIndexOutOfBoundsException: 37
[junit4:junit4] 1> java.lang.ArrayIndexOutOfBoundsException: 37
[junit4:junit4] 1> at org.apache.lucene.index.sorter.SortingAtomicReader$SortingDocsAndPositionsEnum.<init>(SortingAtomicReader.java:454)
[junit4:junit4] 1> at org.apache.lucene.index.sorter.SortingAtomicReader$SortingTermsEnum.docsAndPositions(SortingAtomicReader.java:161)
[junit4:junit4] 1> at org.apache.lucene.index.TermsEnum.docsAndPositions(TermsEnum.java:183)
[junit4:junit4] 1> at org.apache.lucene.index.CheckIndex.checkFields(CheckIndex.java:793)
[junit4:junit4] 1> at org.apache.lucene.index.CheckIndex.testPostings(CheckIndex.java:1200)
[junit4:junit4] 1> at org.apache.lucene.index.CheckIndex.testPostings(CheckIndex.java:1177)
[junit4:junit4] 1> at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:243)
[junit4:junit4] 1> at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:234)
[junit4:junit4] 1> at org.apache.lucene.index.sorter.SortingAtomicReaderTest.beforeClassSortingAtomicReaderTest(SortingAtomicReaderTest.java:72)
[junit4:junit4] 1> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] 1> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] 1> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] 1> at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] 1> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] 1> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 1> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] 1> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] 1> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] 1> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 1> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] 1> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 1> test: stored fields.......OK [67 total field count; avg 1 fields per doc]
[junit4:junit4] 1> test: term vectors........OK [67 total vector count; avg 1 term/freq vector fields per doc]
[junit4:junit4] 1> test: docvalues...........OK [0 total doc count; 4 docvalues fields]
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {positions=DFR I(ne)1, term_vectors=DFR GZ(0.3)}, locale=vi_VN, timezone=Etc/UTC
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=1,free=26787824,total=125829120
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestPKIndexSplitter, TestIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest, SortingAtomicReaderTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SortingAtomicReaderTest -Dtests.seed=ECF2C004E4BBAB07 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=vi_VN -Dtests.timezone=Etc/UTC -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J1 | SortingAtomicReaderTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.RuntimeException: CheckReader failed
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([ECF2C004E4BBAB07]:0)
[junit4:junit4] > at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:255)
[junit4:junit4] > at org.apache.lucene.util._TestUtil.checkReader(_TestUtil.java:234)
[junit4:junit4] > at org.apache.lucene.index.sorter.SortingAtomicReaderTest.beforeClassSortingAtomicReaderTest(SortingAtomicReaderTest.java:72)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:677)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] Completed on J1 in 1.27s, 0 tests, 1 error <<< FAILURES!
[...truncated 14 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build.xml:548: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1758: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/module-build.xml:61: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 8 suites, 21 tests, 2 suite-level errors
Total time: 27 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3797 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3797/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.core.OpenCloseCoreStressTest
Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=2126, name=qtp863626397-2126, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.
Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=2126, name=qtp863626397-2126, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.
at __randomizedtesting.SeedInfo.seed([A54188910C21B76F]:0)
at java.lang.Thread.getStackTrace(Thread.java:1567)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:929)
at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:909)
at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:904)
at org.apache.solr.core.MockFSDirectoryFactory.create(MockFSDirectoryFactory.java:34)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:266)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:222)
at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:124)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1394)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1530)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:784)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:929)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:959)
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1155)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:365)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/OpenCloseCoreStressTest_/00011_core/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d2b6a2b)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
... 1 more
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
No registered leader was found, collection:collection1 slice:shard1
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found, collection:collection1 slice:shard1
at __randomizedtesting.SeedInfo.seed([A54188910C21B76F:24A706897B7ED753]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:481)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:295)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:116)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9197 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T1552 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 6 T1552 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038
[junit4:junit4] 2> 14 T1552 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 15 T1553 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 18 T1553 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 18 T1553 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 18 T1553 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 19 T1553 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 20 T1553 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 116 T1552 oasc.ZkTestServer.run start zk server on port:34904
[junit4:junit4] 2> 116 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7e76a0b3
[junit4:junit4] 2> 117 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 117 T1558 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 118 T1558 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 118 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34905
[junit4:junit4] 2> 119 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34905
[junit4:junit4] 2> 119 T1556 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 121 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0000 with negotiated timeout 10000 for client /140.211.11.196:34905
[junit4:junit4] 2> 122 T1558 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0000, negotiated timeout = 10000
[junit4:junit4] 2> 122 T1559 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e76a0b3 name:ZooKeeperConnection Watcher:127.0.0.1:34904 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 122 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 123 T1552 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 126 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0000
[junit4:junit4] 2> 134 T1559 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 134 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0000 closed
[junit4:junit4] 2> 135 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34905 which had sessionid 0x13d50de710d0000
[junit4:junit4] 2> 135 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1ac39ed5
[junit4:junit4] 2> 136 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 136 T1560 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 137 T1560 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 137 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34906
[junit4:junit4] 2> 137 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34906
[junit4:junit4] 2> 139 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0001 with negotiated timeout 10000 for client /140.211.11.196:34906
[junit4:junit4] 2> 139 T1560 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0001, negotiated timeout = 10000
[junit4:junit4] 2> 139 T1561 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ac39ed5 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 139 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 140 T1552 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 150 T1552 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 153 T1552 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 156 T1552 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 164 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 165 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 177 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 178 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 282 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 283 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 286 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 287 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 290 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 291 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 299 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 299 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 302 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 303 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 307 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 307 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 310 T1552 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 311 T1552 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 314 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0001
[junit4:junit4] 2> 321 T1561 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 321 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34906 which had sessionid 0x13d50de710d0001
[junit4:junit4] 2> 321 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0001 closed
[junit4:junit4] 2> 511 T1552 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 517 T1552 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34907
[junit4:junit4] 2> 517 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 518 T1552 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 518 T1552 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345
[junit4:junit4] 2> 519 T1552 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/solr.xml
[junit4:junit4] 2> 519 T1552 oasc.CoreContainer.<init> New CoreContainer 1627265348
[junit4:junit4] 2> 520 T1552 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/'
[junit4:junit4] 2> 521 T1552 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/'
[junit4:junit4] 2> 583 T1552 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 584 T1552 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 585 T1552 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 585 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 586 T1552 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 586 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 587 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 587 T1552 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 588 T1552 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 588 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 611 T1552 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 631 T1552 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:34904/solr
[junit4:junit4] 2> 632 T1552 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 632 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@314d3ece
[junit4:junit4] 2> 633 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 634 T1571 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 635 T1571 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 635 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34908
[junit4:junit4] 2> 635 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34908
[junit4:junit4] 2> 638 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0002 with negotiated timeout 20000 for client /140.211.11.196:34908
[junit4:junit4] 2> 638 T1571 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0002, negotiated timeout = 20000
[junit4:junit4] 2> 638 T1572 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@314d3ece name:ZooKeeperConnection Watcher:127.0.0.1:34904 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 638 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 640 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0002
[junit4:junit4] 2> 650 T1572 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 650 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34908 which had sessionid 0x13d50de710d0002
[junit4:junit4] 2> 650 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0002 closed
[junit4:junit4] 2> 651 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 663 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@13fe2703
[junit4:junit4] 2> 664 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 665 T1573 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 665 T1573 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 665 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34909
[junit4:junit4] 2> 666 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34909
[junit4:junit4] 2> 667 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0003 with negotiated timeout 20000 for client /140.211.11.196:34909
[junit4:junit4] 2> 667 T1573 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0003, negotiated timeout = 20000
[junit4:junit4] 2> 668 T1574 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13fe2703 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 668 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 695 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 768 T1552 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 781 T1552 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34907_
[junit4:junit4] 2> 782 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34907_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34907_
[junit4:junit4] 2> 803 T1552 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34907_
[junit4:junit4] 2> 814 T1552 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 866 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 878 T1552 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 896 T1552 oasc.Overseer.start Overseer (id=89316483926917123-127.0.0.1:34907_-n_0000000000) starting
[junit4:junit4] 2> 896 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 898 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 952 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 954 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 992 T1576 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 993 T1552 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 996 T1552 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 998 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1001 T1575 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1006 T1577 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/collection1
[junit4:junit4] 2> 1006 T1577 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1006 T1577 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1007 T1577 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1008 T1577 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/collection1/'
[junit4:junit4] 2> 1009 T1577 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/collection1/lib/README' to classloader
[junit4:junit4] 2> 1010 T1577 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1061 T1577 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1128 T1577 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1229 T1577 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 1236 T1577 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1900 T1577 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1916 T1577 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1921 T1577 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1935 T1577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1940 T1577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1944 T1577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1945 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1946 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1947 T1577 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1948 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1948 T1577 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1948 T1577 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362861388345/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data/
[junit4:junit4] 2> 1949 T1577 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ec742a8
[junit4:junit4] 2> 1949 T1577 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1950 T1577 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data forceNew: false
[junit4:junit4] 2> 1950 T1577 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data/index/
[junit4:junit4] 2> 1951 T1577 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1951 T1577 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data/index forceNew: false
[junit4:junit4] 2> 1955 T1577 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1955 T1577 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1958 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1958 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1959 T1577 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1959 T1577 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1960 T1577 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1960 T1577 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1960 T1577 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1961 T1577 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1961 T1577 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1974 T1577 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1980 T1577 oass.SolrIndexSearcher.<init> Opening Searcher@61022955 main
[junit4:junit4] 2> 1980 T1577 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/control/data/tlog
[junit4:junit4] 2> 1981 T1577 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1982 T1577 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1985 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@61022955 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1987 T1577 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1987 T1577 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1988 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 2506 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2507 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:34907_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34907"}
[junit4:junit4] 2> 2507 T1575 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2507 T1575 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 2508 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 2537 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3005 T1577 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 3005 T1577 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34907 collection:control_collection shard:shard1
[junit4:junit4] 2> 3006 T1577 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 3022 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 3023 T1577 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 3023 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3025 T1577 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 3026 T1577 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 3026 T1577 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34907/collection1/
[junit4:junit4] 2> 3026 T1577 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 3026 T1577 oasc.SyncStrategy.syncToMe http://127.0.0.1:34907/collection1/ has no replicas
[junit4:junit4] 2> 3027 T1577 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34907/collection1/
[junit4:junit4] 2> 3027 T1577 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 3040 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4042 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4096 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4113 T1577 oasc.ZkController.register We are http://127.0.0.1:34907/collection1/ and leader is http://127.0.0.1:34907/collection1/
[junit4:junit4] 2> 4113 T1577 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34907
[junit4:junit4] 2> 4113 T1577 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 4113 T1577 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4114 T1577 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4115 T1577 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4116 T1552 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4117 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4118 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4134 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 4135 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@447ac5cc
[junit4:junit4] 2> 4136 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4136 T1580 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4137 T1580 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 4137 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34913
[junit4:junit4] 2> 4138 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34913
[junit4:junit4] 2> 4139 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0004 with negotiated timeout 10000 for client /140.211.11.196:34913
[junit4:junit4] 2> 4139 T1580 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0004, negotiated timeout = 10000
[junit4:junit4] 2> 4139 T1581 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@447ac5cc name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4139 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4141 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4143 T1552 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4335 T1552 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4338 T1552 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34914
[junit4:junit4] 2> 4339 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4340 T1552 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4340 T1552 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168
[junit4:junit4] 2> 4341 T1552 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/solr.xml
[junit4:junit4] 2> 4341 T1552 oasc.CoreContainer.<init> New CoreContainer 677062892
[junit4:junit4] 2> 4342 T1552 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/'
[junit4:junit4] 2> 4343 T1552 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/'
[junit4:junit4] 2> 4405 T1552 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4406 T1552 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4406 T1552 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4407 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4407 T1552 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4408 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4408 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4409 T1552 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4409 T1552 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4410 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4432 T1552 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4454 T1552 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:34904/solr
[junit4:junit4] 2> 4454 T1552 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4455 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@78ac6d93
[junit4:junit4] 2> 4456 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4456 T1591 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4457 T1591 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 4458 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34915
[junit4:junit4] 2> 4458 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34915
[junit4:junit4] 2> 4459 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0005 with negotiated timeout 20000 for client /140.211.11.196:34915
[junit4:junit4] 2> 4460 T1591 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0005, negotiated timeout = 20000
[junit4:junit4] 2> 4460 T1592 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78ac6d93 name:ZooKeeperConnection Watcher:127.0.0.1:34904 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4460 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4462 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0005
[junit4:junit4] 2> 4463 T1592 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4463 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34915 which had sessionid 0x13d50de710d0005
[junit4:junit4] 2> 4463 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0005 closed
[junit4:junit4] 2> 4464 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4476 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7affb264
[junit4:junit4] 2> 4477 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4478 T1593 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4478 T1593 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 4479 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34917
[junit4:junit4] 2> 4479 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34917
[junit4:junit4] 2> 4480 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0006 with negotiated timeout 20000 for client /140.211.11.196:34917
[junit4:junit4] 2> 4480 T1593 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0006, negotiated timeout = 20000
[junit4:junit4] 2> 4481 T1594 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7affb264 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4481 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4482 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4483 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4493 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5496 T1552 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34914_
[junit4:junit4] 2> 5498 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34914_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34914_
[junit4:junit4] 2> 5525 T1552 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34914_
[junit4:junit4] 2> 5528 T1581 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5529 T1574 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5529 T1594 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5529 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5541 T1595 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/collection1
[junit4:junit4] 2> 5541 T1595 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 5542 T1595 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 5542 T1595 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 5543 T1595 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/collection1/'
[junit4:junit4] 2> 5544 T1595 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 5545 T1595 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/collection1/lib/README' to classloader
[junit4:junit4] 2> 5596 T1595 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 5602 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 5603 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:34907_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34907"}
[junit4:junit4] 2> 5629 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5629 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5629 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5661 T1595 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5762 T1595 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5769 T1595 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6441 T1595 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6455 T1595 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6459 T1595 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6473 T1595 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6478 T1595 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6483 T1595 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6485 T1595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6486 T1595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6486 T1595 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6487 T1595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6487 T1595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6488 T1595 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1/
[junit4:junit4] 2> 6488 T1595 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ec742a8
[junit4:junit4] 2> 6488 T1595 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6489 T1595 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1 forceNew: false
[junit4:junit4] 2> 6490 T1595 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1/index/
[junit4:junit4] 2> 6490 T1595 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6491 T1595 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1/index forceNew: false
[junit4:junit4] 2> 6494 T1595 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6494 T1595 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6497 T1595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6497 T1595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6498 T1595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6498 T1595 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6499 T1595 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6499 T1595 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6499 T1595 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6500 T1595 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6500 T1595 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6513 T1595 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6519 T1595 oass.SolrIndexSearcher.<init> Opening Searcher@5abd412b main
[junit4:junit4] 2> 6520 T1595 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty1/tlog
[junit4:junit4] 2> 6521 T1595 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6521 T1595 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6525 T1596 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5abd412b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6527 T1595 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 6527 T1595 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 7133 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7134 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34914_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34914"}
[junit4:junit4] 2> 7134 T1575 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 7135 T1575 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 7147 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7147 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7147 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7556 T1595 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7556 T1595 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34914 collection:collection1 shard:shard1
[junit4:junit4] 2> 7557 T1595 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7572 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7573 T1595 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7574 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7584 T1595 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7584 T1595 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7585 T1595 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34914/collection1/
[junit4:junit4] 2> 7585 T1595 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7585 T1595 oasc.SyncStrategy.syncToMe http://127.0.0.1:34914/collection1/ has no replicas
[junit4:junit4] 2> 7585 T1595 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34914/collection1/
[junit4:junit4] 2> 7586 T1595 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7589 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8656 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8693 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8693 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8694 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8722 T1595 oasc.ZkController.register We are http://127.0.0.1:34914/collection1/ and leader is http://127.0.0.1:34914/collection1/
[junit4:junit4] 2> 8722 T1595 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34914
[junit4:junit4] 2> 8722 T1595 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8722 T1595 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8723 T1595 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8724 T1595 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8726 T1552 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 8726 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8727 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8937 T1552 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8941 T1552 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34928
[junit4:junit4] 2> 8942 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8942 T1552 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8943 T1552 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767
[junit4:junit4] 2> 8943 T1552 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/solr.xml
[junit4:junit4] 2> 8944 T1552 oasc.CoreContainer.<init> New CoreContainer 890022350
[junit4:junit4] 2> 8944 T1552 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/'
[junit4:junit4] 2> 8945 T1552 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/'
[junit4:junit4] 2> 9008 T1552 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 9009 T1552 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 9010 T1552 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 9010 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 9011 T1552 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 9011 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 9012 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 9012 T1552 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 9013 T1552 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 9013 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 9036 T1552 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 9056 T1552 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:34904/solr
[junit4:junit4] 2> 9057 T1552 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 9057 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@481e1ce1
[junit4:junit4] 2> 9058 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 9059 T1607 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 9060 T1607 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 9060 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34929
[junit4:junit4] 2> 9060 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34929
[junit4:junit4] 2> 9062 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0007 with negotiated timeout 20000 for client /140.211.11.196:34929
[junit4:junit4] 2> 9062 T1607 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0007, negotiated timeout = 20000
[junit4:junit4] 2> 9063 T1608 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@481e1ce1 name:ZooKeeperConnection Watcher:127.0.0.1:34904 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 9063 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 9064 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0007
[junit4:junit4] 2> 9076 T1608 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 9076 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34929 which had sessionid 0x13d50de710d0007
[junit4:junit4] 2> 9076 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0007 closed
[junit4:junit4] 2> 9077 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 9090 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@40718723
[junit4:junit4] 2> 9091 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 9091 T1609 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 9092 T1609 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 9092 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34930
[junit4:junit4] 2> 9093 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34930
[junit4:junit4] 2> 9094 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0008 with negotiated timeout 20000 for client /140.211.11.196:34930
[junit4:junit4] 2> 9094 T1609 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0008, negotiated timeout = 20000
[junit4:junit4] 2> 9094 T1610 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40718723 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 9094 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 9096 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 9098 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 9101 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 10104 T1552 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34928_
[junit4:junit4] 2> 10105 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34928_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34928_
[junit4:junit4] 2> 10107 T1552 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34928_
[junit4:junit4] 2> 10110 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 10110 T1594 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 10110 T1574 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 10110 T1610 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 10110 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10111 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10112 T1581 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 10123 T1611 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/collection1
[junit4:junit4] 2> 10123 T1611 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 10124 T1611 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 10124 T1611 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 10125 T1611 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/collection1/'
[junit4:junit4] 2> 10126 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/collection1/lib/README' to classloader
[junit4:junit4] 2> 10127 T1611 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 10178 T1611 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 10206 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10207 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34914_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34914"}
[junit4:junit4] 2> 10221 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10221 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10221 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10221 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10241 T1611 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 10342 T1611 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 10349 T1611 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 11002 T1611 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 11011 T1611 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 11013 T1611 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 11025 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 11029 T1611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 11032 T1611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 11033 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 11034 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 11035 T1611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 11036 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 11036 T1611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 11036 T1611 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362861396767/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2/
[junit4:junit4] 2> 11036 T1611 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ec742a8
[junit4:junit4] 2> 11037 T1611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 11037 T1611 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2 forceNew: false
[junit4:junit4] 2> 11038 T1611 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2/index/
[junit4:junit4] 2> 11038 T1611 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 11038 T1611 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2/index forceNew: false
[junit4:junit4] 2> 11040 T1611 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cf1b419 lockFactory=org.apache.lucene.store.NativeFSLockFactory@230cdd79)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 11041 T1611 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 11043 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 11043 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 11044 T1611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 11044 T1611 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 11045 T1611 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 11045 T1611 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 11045 T1611 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 11046 T1611 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 11046 T1611 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 11063 T1611 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 11070 T1611 oass.SolrIndexSearcher.<init> Opening Searcher@7da5f8e4 main
[junit4:junit4] 2> 11070 T1611 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty2/tlog
[junit4:junit4] 2> 11071 T1611 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 11071 T1611 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 11076 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7da5f8e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 11078 T1611 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 11078 T1611 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11726 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11727 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34928_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34928"}
[junit4:junit4] 2> 11728 T1575 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11728 T1575 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11752 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11752 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11752 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11752 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12081 T1611 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 12081 T1611 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34928 collection:collection1 shard:shard2
[junit4:junit4] 2> 12082 T1611 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 12097 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:delete cxid:0x47 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 12098 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 12098 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:create cxid:0x48 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12108 T1611 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 12109 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 12109 T1611 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34928/collection1/
[junit4:junit4] 2> 12109 T1611 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 12109 T1611 oasc.SyncStrategy.syncToMe http://127.0.0.1:34928/collection1/ has no replicas
[junit4:junit4] 2> 12110 T1611 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34928/collection1/
[junit4:junit4] 2> 12110 T1611 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 12113 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0008 type:create cxid:0x51 zxid:0x83 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13262 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13285 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13285 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13285 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13285 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13339 T1611 oasc.ZkController.register We are http://127.0.0.1:34928/collection1/ and leader is http://127.0.0.1:34928/collection1/
[junit4:junit4] 2> 13339 T1611 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34928
[junit4:junit4] 2> 13339 T1611 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 13339 T1611 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 13340 T1611 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13341 T1611 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13343 T1552 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 13343 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 13344 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 13548 T1552 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 13552 T1552 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34938
[junit4:junit4] 2> 13553 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 13553 T1552 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 13554 T1552 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382
[junit4:junit4] 2> 13554 T1552 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/solr.xml
[junit4:junit4] 2> 13555 T1552 oasc.CoreContainer.<init> New CoreContainer 616511326
[junit4:junit4] 2> 13555 T1552 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/'
[junit4:junit4] 2> 13556 T1552 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/'
[junit4:junit4] 2> 13619 T1552 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 13619 T1552 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 13620 T1552 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 13620 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 13621 T1552 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 13621 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 13622 T1552 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 13623 T1552 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 13623 T1552 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 13624 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 13644 T1552 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 13664 T1552 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:34904/solr
[junit4:junit4] 2> 13665 T1552 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 13665 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@fccf013
[junit4:junit4] 2> 13666 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13667 T1623 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13668 T1623 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 13668 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34939
[junit4:junit4] 2> 13668 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34939
[junit4:junit4] 2> 13672 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d0009 with negotiated timeout 20000 for client /140.211.11.196:34939
[junit4:junit4] 2> 13672 T1623 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d0009, negotiated timeout = 20000
[junit4:junit4] 2> 13673 T1624 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fccf013 name:ZooKeeperConnection Watcher:127.0.0.1:34904 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13673 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13674 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d0009
[junit4:junit4] 2> 13685 T1624 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 13685 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34939 which had sessionid 0x13d50de710d0009
[junit4:junit4] 2> 13685 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d0009 closed
[junit4:junit4] 2> 13686 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 13696 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@55e8d906
[junit4:junit4] 2> 13697 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13697 T1625 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13698 T1625 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 13698 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34940
[junit4:junit4] 2> 13698 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34940
[junit4:junit4] 2> 13700 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d000a with negotiated timeout 20000 for client /140.211.11.196:34940
[junit4:junit4] 2> 13700 T1625 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d000a, negotiated timeout = 20000
[junit4:junit4] 2> 13700 T1626 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55e8d906 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13700 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13701 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d000a type:create cxid:0x1 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13703 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d000a type:create cxid:0x2 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13711 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 14715 T1552 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 14716 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d000a type:delete cxid:0xb zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34938_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 14718 T1552 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 14721 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14721 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14721 T1594 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14721 T1626 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14722 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14722 T1574 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14722 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14723 T1581 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14723 T1610 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14734 T1627 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/collection1
[junit4:junit4] 2> 14735 T1627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 14735 T1627 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 14736 T1627 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 14737 T1627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/collection1/'
[junit4:junit4] 2> 14738 T1627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 14739 T1627 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/collection1/lib/README' to classloader
[junit4:junit4] 2> 14789 T1627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 14800 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14800 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34928_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34928"}
[junit4:junit4] 2> 14820 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14820 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14820 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14820 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14820 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14851 T1627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 14952 T1627 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 14975 T1627 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15569 T1627 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15577 T1627 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15580 T1627 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15590 T1627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15594 T1627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15597 T1627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15598 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15599 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15599 T1627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15600 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15600 T1627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15600 T1627 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362861401382/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/
[junit4:junit4] 2> 15601 T1627 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ec742a8
[junit4:junit4] 2> 15601 T1627 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15602 T1627 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3 forceNew: false
[junit4:junit4] 2> 15602 T1627 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/index/
[junit4:junit4] 2> 15602 T1627 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15603 T1627 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/index forceNew: false
[junit4:junit4] 2> 15605 T1627 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15605 T1627 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15607 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15608 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15608 T1627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15609 T1627 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15609 T1627 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15609 T1627 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15609 T1627 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15610 T1627 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15610 T1627 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15618 T1627 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15622 T1627 oass.SolrIndexSearcher.<init> Opening Searcher@5846f634 main
[junit4:junit4] 2> 15622 T1627 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/tlog
[junit4:junit4] 2> 15623 T1627 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15623 T1627 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15626 T1628 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5846f634 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15628 T1627 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15628 T1627 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16332 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16332 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938"}
[junit4:junit4] 2> 16333 T1575 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 16333 T1575 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16353 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16353 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16353 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16353 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16353 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16633 T1627 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16633 T1627 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34938 collection:collection1 shard:shard1
[junit4:junit4] 2> 16636 T1627 oasc.ZkController.register We are http://127.0.0.1:34938/collection1/ and leader is http://127.0.0.1:34914/collection1/
[junit4:junit4] 2> 16637 T1627 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34938
[junit4:junit4] 2> 16637 T1627 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16637 T1627 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C108 name=collection1 org.apache.solr.core.SolrCore@e1c1c19 url=http://127.0.0.1:34938/collection1 node=127.0.0.1:34938_ C108_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34938_, base_url=http://127.0.0.1:34938}
[junit4:junit4] 2> 16638 T1629 C108 P34938 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16638 T1627 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16639 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16639 T1629 C108 P34938 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16639 T1629 C108 P34938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16640 T1552 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 16640 T1552 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16641 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16656 T1629 C108 P34938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16656 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16657 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16670 T1585 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/onenodecollectioncore
[junit4:junit4] 2> 16671 T1585 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16671 T1585 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16672 T1585 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16672 T1585 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16673 T1585 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16675 T1585 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16677 T1585 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/onenodecollectioncore/'
[junit4:junit4] 2> 16724 T1585 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16765 T1585 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16866 T1585 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16870 T1585 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17311 T1585 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17320 T1585 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17323 T1585 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17335 T1585 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17338 T1585 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17342 T1585 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17343 T1585 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17344 T1585 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17344 T1585 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17345 T1585 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17345 T1585 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17346 T1585 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362861392168/onenodecollectioncore/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection/
[junit4:junit4] 2> 17346 T1585 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6ec742a8
[junit4:junit4] 2> 17346 T1585 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17347 T1585 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection forceNew: false
[junit4:junit4] 2> 17347 T1585 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection/index/
[junit4:junit4] 2> 17348 T1585 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17348 T1585 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection/index forceNew: false
[junit4:junit4] 2> 17350 T1585 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5c604499 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a37e218)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17351 T1585 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17353 T1585 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17353 T1585 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17354 T1585 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17354 T1585 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17355 T1585 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17355 T1585 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17355 T1585 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17356 T1585 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17356 T1585 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17364 T1585 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17368 T1585 oass.SolrIndexSearcher.<init> Opening Searcher@312f638d main
[junit4:junit4] 2> 17369 T1585 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection/tlog
[junit4:junit4] 2> 17370 T1585 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17370 T1585 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17373 T1631 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@312f638d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17373 T1585 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 17857 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17858 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938"}
[junit4:junit4] 2> 17871 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:34914_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34914"}
[junit4:junit4] 2> 17871 T1575 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 17871 T1575 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17898 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17898 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17898 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17898 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17898 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18376 T1585 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 18377 T1585 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:34914 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 18378 T1585 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 18394 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:delete cxid:0xa1 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 18395 T1585 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 18396 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0xa2 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18405 T1585 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 18406 T1585 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 18406 T1585 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34914/onenodecollectioncore/
[junit4:junit4] 2> 18407 T1585 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 18407 T1585 oasc.SyncStrategy.syncToMe http://127.0.0.1:34914/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 18407 T1585 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34914/onenodecollectioncore/
[junit4:junit4] 2> 18408 T1585 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 18412 T1557 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d50de710d0006 type:create cxid:0xac zxid:0xb2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18667 T1584 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=4&state=recovering&nodeName=127.0.0.1:34938_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 19404 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19430 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19430 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19430 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19430 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19430 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19434 T1585 oasc.ZkController.register We are http://127.0.0.1:34914/onenodecollectioncore/ and leader is http://127.0.0.1:34914/onenodecollectioncore/
[junit4:junit4] 2> 19434 T1585 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:34914
[junit4:junit4] 2> 19435 T1585 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 19435 T1585 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 19441 T1585 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19442 T1585 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&roles=none&name=onenodecollectioncore&action=CREATE&collection=onenodecollection&dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362861388038/onenodecollection&wt=javabin&version=2} status=0 QTime=2772
[junit4:junit4] 2> 19443 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 19444 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20446 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C108_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34938_, base_url=http://127.0.0.1:34938}
[junit4:junit4] 2> 20669 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34914/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20670 T1629 C108 P34938 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34938 START replicas=[http://127.0.0.1:34914/collection1/] nUpdates=100
[junit4:junit4] 2> 20671 T1629 C108 P34938 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20671 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20671 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20671 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20671 T1629 C108 P34938 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20672 T1629 C108 P34938 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34914/collection1/. core=collection1
[junit4:junit4] 2> 20672 T1629 C108 P34938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C109 name=collection1 org.apache.solr.core.SolrCore@760c626d url=http://127.0.0.1:34914/collection1 node=127.0.0.1:34914_ C109_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34914_, base_url=http://127.0.0.1:34914, leader=true}
[junit4:junit4] 2> 20679 T1586 C109 P34914 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 20690 T1587 C109 P34914 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20692 T1587 C109 P34914 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20693 T1587 C109 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20694 T1587 C109 P34914 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20695 T1587 C109 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20695 T1587 C109 P34914 oass.SolrIndexSearcher.<init> Opening Searcher@313cc762 realtime
[junit4:junit4] 2> 20696 T1587 C109 P34914 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20696 T1587 C109 P34914 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 20697 T1629 C108 P34938 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20698 T1629 C108 P34938 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20700 T1588 C109 P34914 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20701 T1588 C109 P34914 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 20701 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 20702 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20702 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20704 T1588 C109 P34914 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 20705 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20706 T1629 C108 P34938 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362861388031/jetty3/index.20130310093648729 forceNew: false
[junit4:junit4] 2> 20706 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@49c687a8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ec61000) fullCopy=false
[junit4:junit4] 2> 20709 T1588 C109 P34914 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 20711 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20712 T1629 C108 P34938 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20712 T1629 C108 P34938 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20714 T1629 C108 P34938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20714 T1629 C108 P34938 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20715 T1629 C108 P34938 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20715 T1629 C108 P34938 oass.SolrIndexSearcher.<init> Opening Searcher@3ece46eb main
[junit4:junit4] 2> 20716 T1628 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ece46eb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 20716 T1629 C108 P34938 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@49c687a8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5ec61000)
[junit4:junit4] 2> 20717 T1629 C108 P34938 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20717 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20717 T1629 C108 P34938 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20717 T1629 C108 P34938 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20721 T1629 C108 P34938 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20943 T1575 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20944 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:34914_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34914"}
[junit4:junit4] 2> 20989 T1575 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938"}
[junit4:junit4] 2> 20998 T1626 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20998 T1581 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20998 T1574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20998 T1594 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20998 T1610 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21449 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21450 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 21451 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C110 name=onenodecollectioncore org.apache.solr.core.SolrCore@5f85849 url=http://127.0.0.1:34914/onenodecollectioncore node=127.0.0.1:34914_ C110_STATE=coll:onenodecollection core:onenodecollectioncore props:{shard=shard1, roles=none, state=active, core=onenodecollectioncore, collection=onenodecollection, node_name=127.0.0.1:34914_, base_url=http://127.0.0.1:34914, leader=true}
[junit4:junit4] 2> 21469 T1589 C110 P34914 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5c604499 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a37e218)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21470 T1589 C110 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21472 T1589 C110 P34914 /update {wt=javabin&version=2} {add=[1 (1429063765318434816)]} 0 5
[junit4:junit4] 2> 21478 T1589 C110 P34914 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21481 T1589 C110 P34914 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5c604499 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a37e218)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5c604499 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a37e218)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 21481 T1589 C110 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 21483 T1589 C110 P34914 oass.SolrIndexSearcher.<init> Opening Searcher@3cef95c0 main
[junit4:junit4] 2> 21483 T1589 C110 P34914 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21484 T1631 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@3cef95c0 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21484 T1589 C110 P34914 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 21488 T1589 C110 P34914 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21489 T1552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 21502 T1589 C110 P34914 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C111 name=collection1 org.apache.solr.core.SolrCore@b2e76a3 url=http://127.0.0.1:34907/collection1 node=127.0.0.1:34907_ C111_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:34907_, base_url=http://127.0.0.1:34907, leader=true}
[junit4:junit4] 2> 21513 T1564 C111 P34907 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21514 T1564 C111 P34907 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21519 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[1 (1429063765364572160)]} 0 8
[junit4:junit4] 2> C108_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34938_, base_url=http://127.0.0.1:34938}
[junit4:junit4] 2> 21553 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429063765392883712)]} 0 4
[junit4:junit4] 2> 21554 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1429063765392883712)]} 0 16
[junit4:junit4] 2> ASYNC NEW_CORE C112 name=collection1 org.apache.solr.core.SolrCore@1006b182 url=http://127.0.0.1:34928/collection1 node=127.0.0.1:34928_ C112_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34928_, base_url=http://127.0.0.1:34928, leader=true}
[junit4:junit4] 2> 21555 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[1]} 0 19
[junit4:junit4] 2> 21558 T1564 C111 P34907 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21563 T1564 C111 P34907 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 21564 T1564 C111 P34907 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> 21566 T1564 C111 P34907 oass.SolrIndexSearcher.<init> Opening Searcher@592206a main
[junit4:junit4] 2> 21566 T1564 C111 P34907 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21567 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@592206a main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21568 T1564 C111 P34907 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 21570 T1584 C109 P34914 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21575 T1584 C109 P34914 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> 21576 T1584 C109 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> 21578 T1584 C109 P34914 oass.SolrIndexSearcher.<init> Opening Searcher@7be53270 main
[junit4:junit4] 2> 21579 T1596 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7be53270 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21623 T1584 C109 P34914 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21624 T1584 C109 P34914 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34938/collection1/, StdNode: http://127.0.0.1:34928/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 21626 T1616 C108 P34938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21627 T1603 C112 P34928 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21632 T1616 C108 P34938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> 21633 T1616 C108 P34938 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> 21635 T1616 C108 P34938 oass.SolrIndexSearcher.<init> Opening Searcher@51795e7b main
[junit4:junit4] 2> 21636 T1628 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51795e7b main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21658 T1616 C108 P34938 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21659 T1616 C108 P34938 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 33
[junit4:junit4] 2> 21659 T1603 C112 P34928 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cf1b419 lockFactory=org.apache.lucene.store.NativeFSLockFactory@230cdd79)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21660 T1603 C112 P34928 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21660 T1603 C112 P34928 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cf1b419 lockFactory=org.apache.lucene.store.NativeFSLockFactory@230cdd79)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@1cf1b419 lockFactory=org.apache.lucene.store.NativeFSLockFactory@230cdd79)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 21661 T1603 C112 P34928 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 21661 T1603 C112 P34928 oass.SolrIndexSearcher.<init> Opening Searcher@38f9f930 main
[junit4:junit4] 2> 21662 T1603 C112 P34928 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21663 T1612 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38f9f930 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 21663 T1603 C112 P34928 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 37
[junit4:junit4] 2> 21664 T1584 C109 P34914 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 94
[junit4:junit4] 2> 21665 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21666 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21667 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 21671 T1564 C111 P34907 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21672 T1552 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:34904/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2356db89
[junit4:junit4] 2> 21673 T1552 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 21673 T1638 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:34904. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 21674 T1638 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:34904, initiating session
[junit4:junit4] 2> 21674 T1554 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34968
[junit4:junit4] 2> 21675 T1554 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34968
[junit4:junit4] 2> 21679 T1556 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d50de710d000b with negotiated timeout 10000 for client /140.211.11.196:34968
[junit4:junit4] 2> 21679 T1638 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:34904, sessionid = 0x13d50de710d000b, negotiated timeout = 10000
[junit4:junit4] 2> 21680 T1639 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2356db89 name:ZooKeeperConnection Watcher:127.0.0.1:34904/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 21680 T1552 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 21682 T1552 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 21685 T1557 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d50de710d000b
[junit4:junit4] 2> 21692 T1639 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21692 T1554 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34968 which had sessionid 0x13d50de710d000b
[junit4:junit4] 2> 21692 T1552 oaz.ZooKeeper.close Session: 0x13d50de710d000b closed
[junit4:junit4] 2> 21695 T1587 C109 P34914 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21698 T1604 C112 P34928 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4:junit4] 2> 21704 T1642 C109 P34914 REQ /select {NOW=1362861409725&shard.url=127.0.0.1:34914/collection1/|127.0.0.1:34938/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21704 T1605 C112 P34928 REQ /select {NOW=1362861409725&shard.url=127.0.0.1:34928/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21707 T1642 C109 P34914 REQ /select {NOW=1362861409725&shard.url=127.0.0.1:34914/collection1/|127.0.0.1:34938/collection1/&ids=1&q=*:*&distrib=false&wt=javabin&isShard=true&version=2} status=0 QTime=0
[junit4:junit4] 2> 21708 T1602 C112 P34928 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=7
[junit4:junit4] 2> 21715 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[2 (1429063765575335936)]} 0 4
[junit4:junit4] 2> 21722 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[2 (1429063765581627392)]} 0 3
[junit4:junit4] 2> 21726 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[3 (1429063765587918848)]} 0 2
[junit4:junit4] 2> 21731 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[3 (1429063765593161728)]} 0 2
[junit4:junit4] 2> 21736 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[4 (1429063765598404608)]} 0 2
[junit4:junit4] 2> 21749 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1429063765603647488)]} 0 4
[junit4:junit4] 2> 21750 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[4 (1429063765603647488)]} 0 11
[junit4:junit4] 2> 21755 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[5 (1429063765618327552)]} 0 2
[junit4:junit4] 2> 21764 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1429063765626716160)]} 0 3
[junit4:junit4] 2> 21765 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[5]} 0 7
[junit4:junit4] 2> 21770 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[6 (1429063765634056192)]} 0 2
[junit4:junit4] 2> 21778 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1429063765642444800)]} 0 2
[junit4:junit4] 2> 21779 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[6]} 0 6
[junit4:junit4] 2> 21785 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[7 (1429063765648736256)]} 0 3
[junit4:junit4] 2> 21794 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1429063765658173440)]} 0 3
[junit4:junit4] 2> 21795 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[7]} 0 7
[junit4:junit4] 2> 21802 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[8 (1429063765666562048)]} 0 4
[junit4:junit4] 2> 21814 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1429063765672853504)]} 0 4
[junit4:junit4] 2> 21815 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[8 (1429063765672853504)]} 0 10
[junit4:junit4] 2> 21820 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[9 (1429063765686484992)]} 0 2
[junit4:junit4] 2> 21831 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1429063765696970752)]} 0 4
[junit4:junit4] 2> 21832 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[9]} 0 8
[junit4:junit4] 2> 21838 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[10 (1429063765704310784)]} 0 3
[junit4:junit4] 2> 21850 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1429063765710602240)]} 0 4
[junit4:junit4] 2> 21851 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[10 (1429063765710602240)]} 0 10
[junit4:junit4] 2> 21856 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[11 (1429063765724233728)]} 0 2
[junit4:junit4] 2> 21868 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1429063765729476608)]} 0 4
[junit4:junit4] 2> 21869 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[11 (1429063765729476608)]} 0 10
[junit4:junit4] 2> 21874 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[12 (1429063765743108096)]} 0 2
[junit4:junit4] 2> 21889 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1429063765752545280)]} 0 4
[junit4:junit4] 2> 21890 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1429063765752545280)]} 0 10
[junit4:junit4] 2> 21891 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[12]} 0 14
[junit4:junit4] 2> 21895 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[13 (1429063765765128192)]} 0 2
[junit4:junit4] 2> 21910 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1429063765773516800)]} 0 4
[junit4:junit4] 2> 21911 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1429063765773516800)]} 0 10
[junit4:junit4] 2> 21912 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[13]} 0 14
[junit4:junit4] 2> 21917 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[14 (1429063765789245440)]} 0 2
[junit4:junit4] 2> 21928 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1429063765794488320)]} 0 3
[junit4:junit4] 2> 21929 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[14 (1429063765794488320)]} 0 9
[junit4:junit4] 2> 21934 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[15 (1429063765806022656)]} 0 2
[junit4:junit4] 2> 21944 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1429063765811265536)]} 0 3
[junit4:junit4] 2> 21945 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[15 (1429063765811265536)]} 0 8
[junit4:junit4] 2> 21950 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[16 (1429063765822799872)]} 0 2
[junit4:junit4] 2> 21963 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1429063765831188480)]} 0 3
[junit4:junit4] 2> 21964 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1429063765831188480)]} 0 8
[junit4:junit4] 2> 21965 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[16]} 0 12
[junit4:junit4] 2> 21973 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[17 (1429063765844819968)]} 0 5
[junit4:junit4] 2> 21981 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[17 (1429063765853208576)]} 0 5
[junit4:junit4] 2> 21986 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[100 (1429063765860548608)]} 0 2
[junit4:junit4] 2> 21999 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[100 (1429063765868937216)]} 0 3
[junit4:junit4] 2> 22000 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[100 (1429063765868937216)]} 0 8
[junit4:junit4] 2> 22001 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[100]} 0 12
[junit4:junit4] 2> 22006 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[101 (1429063765881520128)]} 0 2
[junit4:junit4] 2> 22016 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[101 (1429063765886763008)]} 0 3
[junit4:junit4] 2> 22017 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[101 (1429063765886763008)]} 0 8
[junit4:junit4] 2> 22022 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[102 (1429063765898297344)]} 0 2
[junit4:junit4] 2> 22030 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[102 (1429063765902491648)]} 0 2
[junit4:junit4] 2> 22031 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[102 (1429063765902491648)]} 0 7
[junit4:junit4] 2> 22035 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[103 (1429063765912977408)]} 0 1
[junit4:junit4] 2> 22040 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[103 (1429063765918220288)]} 0 2
[junit4:junit4] 2> 22046 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[104 (1429063765923463168)]} 0 2
[junit4:junit4] 2> 22053 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[104 (1429063765930803200)]} 0 2
[junit4:junit4] 2> 22053 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[104]} 0 5
[junit4:junit4] 2> 22058 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[105 (1429063765936046080)]} 0 2
[junit4:junit4] 2> 22063 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[105 (1429063765941288960)]} 0 2
[junit4:junit4] 2> 22068 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[106 (1429063765946531840)]} 0 2
[junit4:junit4] 2> 22074 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[106 (1429063765952823296)]} 0 2
[junit4:junit4] 2> 22074 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[106]} 0 4
[junit4:junit4] 2> 22079 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[107 (1429063765958066176)]} 0 2
[junit4:junit4] 2> 22089 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[107 (1429063765963309056)]} 0 3
[junit4:junit4] 2> 22090 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[107 (1429063765963309056)]} 0 8
[junit4:junit4] 2> 22095 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[108 (1429063765974843392)]} 0 2
[junit4:junit4] 2> 22106 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[108 (1429063765981134848)]} 0 3
[junit4:junit4] 2> 22107 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[108 (1429063765981134848)]} 0 9
[junit4:junit4] 2> 22113 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[109 (1429063765993717760)]} 0 2
[junit4:junit4] 2> 22123 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[109 (1429063765998960640)]} 0 3
[junit4:junit4] 2> 22124 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[109 (1429063765998960640)]} 0 8
[junit4:junit4] 2> 22129 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[110 (1429063766010494976)]} 0 2
[junit4:junit4] 2> 22138 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[110 (1429063766015737856)]} 0 2
[junit4:junit4] 2> 22139 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[110 (1429063766015737856)]} 0 7
[junit4:junit4] 2> 22144 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[111 (1429063766026223616)]} 0 2
[junit4:junit4] 2> 22154 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[111 (1429063766031466496)]} 0 3
[junit4:junit4] 2> 22155 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[111 (1429063766031466496)]} 0 8
[junit4:junit4] 2> 22160 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[112 (1429063766043000832)]} 0 2
[junit4:junit4] 2> 22165 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[112 (1429063766049292288)]} 0 2
[junit4:junit4] 2> 22170 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[113 (1429063766053486592)]} 0 2
[junit4:junit4] 2> 22180 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[113 (1429063766058729472)]} 0 3
[junit4:junit4] 2> 22180 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[113 (1429063766058729472)]} 0 7
[junit4:junit4] 2> 22185 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[114 (1429063766069215232)]} 0 2
[junit4:junit4] 2> 22197 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[114 (1429063766076555264)]} 0 3
[junit4:junit4] 2> 22197 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[114 (1429063766076555264)]} 0 7
[junit4:junit4] 2> 22198 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[114]} 0 11
[junit4:junit4] 2> 22203 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[115 (1429063766088089600)]} 0 2
[junit4:junit4] 2> 22207 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[115 (1429063766092283904)]} 0 2
[junit4:junit4] 2> 22211 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[116 (1429063766097526784)]} 0 1
[junit4:junit4] 2> 22215 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[116 (1429063766101721088)]} 0 1
[junit4:junit4] 2> 22220 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[117 (1429063766105915392)]} 0 2
[junit4:junit4] 2> 22225 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[117 (1429063766111158272)]} 0 2
[junit4:junit4] 2> 22229 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[118 (1429063766115352576)]} 0 2
[junit4:junit4] 2> 22239 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[118 (1429063766120595456)]} 0 3
[junit4:junit4] 2> 22240 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[118 (1429063766120595456)]} 0 8
[junit4:junit4] 2> 22244 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[119 (1429063766131081216)]} 0 2
[junit4:junit4] 2> 22251 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[119 (1429063766138421248)]} 0 2
[junit4:junit4] 2> 22252 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[119]} 0 6
[junit4:junit4] 2> 22257 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[120 (1429063766144712704)]} 0 2
[junit4:junit4] 2> 22260 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[120 (1429063766148907008)]} 0 1
[junit4:junit4] 2> 22264 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[121 (1429063766153101312)]} 0 1
[junit4:junit4] 2> 22268 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[121 (1429063766157295616)]} 0 1
[junit4:junit4] 2> 22273 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[122 (1429063766161489920)]} 0 2
[junit4:junit4] 2> 22285 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[122 (1429063766168829952)]} 0 3
[junit4:junit4] 2> 22286 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[122 (1429063766168829952)]} 0 8
[junit4:junit4] 2> 22286 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[122]} 0 11
[junit4:junit4] 2> 22291 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[123 (1429063766180364288)]} 0 2
[junit4:junit4] 2> 22299 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[123 (1429063766184558592)]} 0 2
[junit4:junit4] 2> 22301 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[123 (1429063766184558592)]} 0 8
[junit4:junit4] 2> 22305 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[124 (1429063766195044352)]} 0 2
[junit4:junit4] 2> 22316 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[124 (1429063766201335808)]} 0 3
[junit4:junit4] 2> 22317 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[124 (1429063766201335808)]} 0 8
[junit4:junit4] 2> 22317 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[124]} 0 10
[junit4:junit4] 2> 22322 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[125 (1429063766212870144)]} 0 2
[junit4:junit4] 2> 22332 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[125 (1429063766219161600)]} 0 3
[junit4:junit4] 2> 22333 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[125 (1429063766219161600)]} 0 8
[junit4:junit4] 2> 22338 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[126 (1429063766229647360)]} 0 2
[junit4:junit4] 2> 22345 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[126 (1429063766236987392)]} 0 2
[junit4:junit4] 2> 22345 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[126]} 0 5
[junit4:junit4] 2> 22349 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[127 (1429063766242230272)]} 0 1
[junit4:junit4] 2> 22356 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[127 (1429063766249570304)]} 0 1
[junit4:junit4] 2> 22357 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[127]} 0 5
[junit4:junit4] 2> 22362 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[128 (1429063766254813184)]} 0 1
[junit4:junit4] 2> 22365 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[128 (1429063766259007488)]} 0 1
[junit4:junit4] 2> 22369 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[129 (1429063766263201792)]} 0 1
[junit4:junit4] 2> 22382 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[129 (1429063766270541824)]} 0 3
[junit4:junit4] 2> 22383 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[129 (1429063766270541824)]} 0 8
[junit4:junit4] 2> 22384 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[129]} 0 12
[junit4:junit4] 2> 22388 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[130 (1429063766283124736)]} 0 1
[junit4:junit4] 2> 22396 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[130 (1429063766290464768)]} 0 2
[junit4:junit4] 2> 22396 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[130]} 0 5
[junit4:junit4] 2> 22400 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[131 (1429063766295707648)]} 0 1
[junit4:junit4] 2> 22408 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[131 (1429063766303047680)]} 0 2
[junit4:junit4] 2> 22409 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[131]} 0 6
[junit4:junit4] 2> 22413 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[132 (1429063766309339136)]} 0 1
[junit4:junit4] 2> 22422 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[132 (1429063766313533440)]} 0 2
[junit4:junit4] 2> 22422 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[132 (1429063766313533440)]} 0 6
[junit4:junit4] 2> 22427 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[133 (1429063766322970624)]} 0 2
[junit4:junit4] 2> 22439 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[133 (1429063766331359232)]} 0 2
[junit4:junit4] 2> 22458 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[133 (1429063766331359232)]} 0 25
[junit4:junit4] 2> 22459 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[133]} 0 29
[junit4:junit4] 2> 22464 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[134 (1429063766362816512)]} 0 2
[junit4:junit4] 2> 22469 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[134 (1429063766367010816)]} 0 2
[junit4:junit4] 2> 22474 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[135 (1429063766372253696)]} 0 2
[junit4:junit4] 2> 22487 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[135 (1429063766380642304)]} 0 3
[junit4:junit4] 2> 22488 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[135 (1429063766380642304)]} 0 8
[junit4:junit4] 2> 22489 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[135]} 0 12
[junit4:junit4] 2> 22494 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[136 (1429063766393225216)]} 0 2
[junit4:junit4] 2> 22499 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[136 (1429063766398468096)]} 0 2
[junit4:junit4] 2> 22504 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[137 (1429063766403710976)]} 0 2
[junit4:junit4] 2> 22512 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[137 (1429063766412099584)]} 0 2
[junit4:junit4] 2> 22512 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[137]} 0 5
[junit4:junit4] 2> 22518 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[138 (1429063766418391040)]} 0 2
[junit4:junit4] 2> 22530 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[138 (1429063766427828224)]} 0 2
[junit4:junit4] 2> 22531 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[138 (1429063766427828224)]} 0 7
[junit4:junit4] 2> 22532 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[138]} 0 11
[junit4:junit4] 2> 22536 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[139 (1429063766438313984)]} 0 1
[junit4:junit4] 2> 22541 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[139 (1429063766442508288)]} 0 2
[junit4:junit4] 2> 22545 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[140 (1429063766447751168)]} 0 1
[junit4:junit4] 2> 22553 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[140 (1429063766455091200)]} 0 2
[junit4:junit4] 2> 22553 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[140]} 0 5
[junit4:junit4] 2> 22558 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[141 (1429063766460334080)]} 0 2
[junit4:junit4] 2> 22562 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[141 (1429063766464528384)]} 0 2
[junit4:junit4] 2> 22567 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[142 (1429063766469771264)]} 0 2
[junit4:junit4] 2> 22575 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[142 (1429063766479208448)]} 0 2
[junit4:junit4] 2> 22576 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[142]} 0 6
[junit4:junit4] 2> 22582 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[143 (1429063766484451328)]} 0 3
[junit4:junit4] 2> 22589 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[143 (1429063766493888512)]} 0 1
[junit4:junit4] 2> 22590 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[143]} 0 5
[junit4:junit4] 2> 22594 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[144 (1429063766499131392)]} 0 1
[junit4:junit4] 2> 22601 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[144 (1429063766506471424)]} 0 1
[junit4:junit4] 2> 22602 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[144]} 0 5
[junit4:junit4] 2> 22606 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[145 (1429063766511714304)]} 0 1
[junit4:junit4] 2> 22613 T1603 C112 P34928 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[145 (1429063766519054336)]} 0 1
[junit4:junit4] 2> 22614 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[145]} 0 5
[junit4:junit4] 2> 22618 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[146 (1429063766524297216)]} 0 1
[junit4:junit4] 2> 22629 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[146 (1429063766531637248)]} 0 2
[junit4:junit4] 2> 22630 T1590 C109 P34914 /update {distrib.from=http://127.0.0.1:34928/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[146 (1429063766531637248)]} 0 6
[junit4:junit4] 2> 22630 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[146]} 0 9
[junit4:junit4] 2> 22634 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[147 (1429063766541074432)]} 0 1
[junit4:junit4] 2> 22638 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[147 (1429063766545268736)]} 0 1
[junit4:junit4] 2> 22642 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[148 (1429063766549463040)]} 0 1
[junit4:junit4] 2> 22649 T1616 C108 P34938 /update {distrib.from=http://127.0.0.1:34914/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[148 (1429063766553657344)]} 0 1
[junit4:junit4] 2> 22650 T1584 C109 P34914 /update {wt=javabin&version=2} {add=[148 (1429063766553657344)]} 0 5
[junit4:junit4] 2> 22654 T1564 C111 P34907 /update {wt=javabin&version=2} {add=[149 (1429063766562045952)]} 0 1
[junit4:junit4] 2> 22658 T1602 C112 P34928 /update {wt=javabin&version=2} {add=[149 (1429063766566240256)]} 0 1
[junit4:junit4] 2> 22659 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 22660 T1552 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22661 T1552 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 22664 T1564 C111 P34907 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22699 T1564 C111 P34907 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_2,generation=2,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, segments_2, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, _0_dv.cfe]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@6b8eca9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@58b82d62)),segFN=segments_3,generation=3,filenames=[_1_nrm.cfs, _1_Lucene41_0.doc, _0.fnm, _0_Lucene41_0.pos, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_dv.cfs, _0_nrm.cfs, _1_dv.cfe, _1.fnm, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1.fdt, _0_Lucene41_0.doc, _1.si, _0_nrm.cfe, _1_dv.cfs, _0.fdx, _0.si, _0_Lucene41_0.tim, segments_3, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 22700 T1564 C111 P34907 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_1_nrm.cfs, _1_Lucene41_0.doc, _0.fnm, _0_Lucene41_0.pos, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_dv.cfs, _0_nrm.cfs, _1_dv.cfe, _1.fnm, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1.fdt, _0_Lucene41_0.doc, _1.si, _0_nrm.cfe, _1_dv.cfs, _0.fdx, _0.si, _0_Lucene41_0.tim, segments_3, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe]
[junit4:junit4] 2> 22702 T1564 C111 P34907 oass.SolrIndexSearcher.<init> Opening Searcher@36ed7357 main
[junit4:junit4] 2> 22703 T1564 C111 P34907 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22703 T1578 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36ed7357 main{StandardDirectoryReader(segments_3:5 _0(5.0):C1 _1(5.0):C66)}
[junit4:junit4] 2> 22704 T1564 C111 P34907 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 40
[junit4:junit4] 2> 22707 T1584 C109 P34914 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22726 T1584 C109 P34914 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5667549f lockFactory=org.apache.lucene.store.NativeFSLockFactory@78ba36cd)),segFN=segments_4,generation=4,filenames=[_1_nrm.cfs, _1_Lucene41_0.doc, _0.fnm, _0_Lucene41_0.pos, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_dv.cfs, _0_nrm.cfs, _1_dv.cfe, _1.fnm, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1.fdt, _0_Lucene41_0.doc, _1.si, _0_nrm.cfe, _1_dv.cfs, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe, segments_4]
[junit4:junit4] 2> 22727 T1584 C109 P34914 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_1_nrm.cfs, _1_Lucene41_0.doc, _0.fnm, _0_Lucene41_0.pos, _1_Lucene41_0.tim, _1_Lucene41_0.tip, _0_dv.cfs, _0_nrm.cfs, _1_dv.cfe, _1.fnm, _1_nrm.cfe, _1.fdx, _1_Lucene41_0.pos, _1.fdt, _0_Lucene41_0.doc, _1.si, _0_nrm.cfe, _1_dv.cfs, _0.fdx, _0.si, _0_Lucene41_0.tim, _0_Lucene41_0.tip, _0.fdt, _0_dv.cfe, segments_4]
[junit4:junit4] 2> 22728 T1584 C109 P34914 oass.SolrIndexSearcher.<init> Opening Searcher@55d9e8d8 main
[junit4:junit4] 2> 22729 T1584 C109 P34914 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22730 T1596 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55d9e8d8 main{StandardDirectoryReader(segments_4:5:nrt _0(5.0):C1 _1(5.0):C31)}
[junit4:junit4] 2> 22730 T1584 C109 P34914 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34938/collection1/, StdNode: http://127.0.0.1:34928/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 22733 T1603 C112 P34928 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22733 T1616 C108 P34938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22754 T1616 C108 P34938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea98d9c lockFactory=org.apache.lucene.store.NativeFSLockFactory@22748141)),segFN=segments_3,generation=3,filenames=[_0.fnm, _0_Lucene41_0.doc, _0_Lucene41_0.pos, _0_nrm.cfe, _0.fdx, _0_dv.cfs, _0_nrm.cfs, _0.si, _0_Lucene41_0.tim, _0.fdt, _0_Lucene41_0.tip, segments_3, _0_dv.cfe]
[junit4:junit4] 2> commit{dir=MockDirWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@4ea
[...truncated too long message...]
eekly)
[junit4:junit4] 2> 88746 T1954 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {id=PostingsFormat(name=SimpleText), text=PostingsFormat(name=MockRandom)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=el, timezone=Pacific/Tarawa
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=1,free=108806856,total=534773760
[junit4:junit4] 2> NOTE: All tests run in this JVM: [CircularListTest, DistributedTermsComponentTest, TestStressRecovery, SearchHandlerTest, RegexBoostProcessorTest, UnloadDistributedZkTest, TestRemoteStreaming, TestXIncludeConfig, TestGroupingSearch, SliceStateTest, BinaryUpdateRequestHandlerTest, TestPhraseSuggestions, TestQueryUtils, PeerSyncTest, QueryElevationComponentTest, TermVectorComponentTest, TestQuerySenderListener, TestFaceting, TestSolrIndexConfig, TestStressVersions, TestDocumentBuilder, TestSolrDeletionPolicy1, AlternateDirectoryTest, TestSurroundQueryParser, SliceStateUpdateTest, TestSweetSpotSimilarityFactory, TestPHPSerializedResponseWriter, TestLFUCache, SoftAutoCommitTest, PrimitiveFieldTypeTest, SuggesterFSTTest, BasicDistributedZkTest, UUIDFieldTest, SpatialFilterTest, TestFieldTypeCollectionResource, ZkCLITest, JsonLoaderTest, SOLR749Test, CoreAdminHandlerTest, TestPropInjectDefaults, CSVRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestIBSimilarityFactory, TestCoreContainer, DirectSolrConnectionTest, TestPerFieldSimilarity, IndexSchemaTest, SchemaVersionSpecificBehaviorTest, TestReversedWildcardFilterFactory, MinimalSchemaTest, TestSolrDeletionPolicy2, ClusterStateUpdateTest, TestJoin, DistributedQueryElevationComponentTest, DocValuesTest, TestFastWriter, TestDocSet, TestCharFilters, TestSolrCoreProperties, LegacyHTMLStripCharFilterTest, TestFieldCollectionResource, AutoCommitTest, TestConfig, StatelessScriptUpdateProcessorFactoryTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OpenCloseCoreStressTest -Dtests.seed=A54188910C21B76F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el -Dtests.timezone=Pacific/Tarawa -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | OpenCloseCoreStressTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=2126, name=qtp863626397-2126, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest], registration stack trace below.
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([A54188910C21B76F]:0)
[junit4:junit4] > at java.lang.Thread.getStackTrace(Thread.java:1567)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:929)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:909)
[junit4:junit4] > at org.apache.lucene.util.LuceneTestCase.newFSDirectory(LuceneTestCase.java:904)
[junit4:junit4] > at org.apache.solr.core.MockFSDirectoryFactory.create(MockFSDirectoryFactory.java:34)
[junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:266)
[junit4:junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:222)
[junit4:junit4] > at org.apache.solr.search.SolrIndexSearcher.<init>(SolrIndexSearcher.java:124)
[junit4:junit4] > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1394)
[junit4:junit4] > at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1530)
[junit4:junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:784)
[junit4:junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
[junit4:junit4] > at org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:929)
[junit4:junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:959)
[junit4:junit4] > at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1155)
[junit4:junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:226)
[junit4:junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
[junit4:junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
[junit4:junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4] > at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
[junit4:junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
[junit4:junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
[junit4:junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
[junit4:junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:365)
[junit4:junit4] > at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
[junit4:junit4] > at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
[junit4:junit4] > at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
[junit4:junit4] > at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
[junit4:junit4] > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
[junit4:junit4] > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4] > at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4] > at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/OpenCloseCoreStressTest_/00011_core/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3d2b6a2b)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > ... 1 more
[junit4:junit4] Completed on J0 in 88.79s, 6 tests, 1 error, 4 skipped <<< FAILURES!
[...truncated 412 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 269 suites, 1137 tests, 1 suite-level error, 1 error, 27 ignored (8 assumptions)
Total time: 48 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3796 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3796/
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
No registered leader was found, collection:collection1 slice:shard1
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found, collection:collection1 slice:shard1
at __randomizedtesting.SeedInfo.seed([6BEF84C45B4C89BB:EA090ADC2C13E987]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:481)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:295)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:116)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9215 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T2106 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/z
[junit4:junit4] 2> 6 T2106 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843
[junit4:junit4] 2> 7 T2106 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 8 T2107 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 11 T2107 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 11 T2107 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 11 T2107 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 12 T2107 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 13 T2107 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 109 T2106 oasc.ZkTestServer.run start zk server on port:11785
[junit4:junit4] 2> 109 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2da3c4e
[junit4:junit4] 2> 110 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 111 T2112 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 111 T2112 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 112 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:21068
[junit4:junit4] 2> 112 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:21068
[junit4:junit4] 2> 113 T2110 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 129 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0000 with negotiated timeout 10000 for client /140.211.11.196:21068
[junit4:junit4] 2> 129 T2112 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0000, negotiated timeout = 10000
[junit4:junit4] 2> 130 T2113 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2da3c4e name:ZooKeeperConnection Watcher:127.0.0.1:11785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 130 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 130 T2106 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 134 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0000
[junit4:junit4] 2> 144 T2113 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 144 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0000 closed
[junit4:junit4] 2> 145 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21068 which had sessionid 0x13d4ea2f76a0000
[junit4:junit4] 2> 145 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3449cae
[junit4:junit4] 2> 146 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 147 T2114 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 147 T2114 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 147 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:26453
[junit4:junit4] 2> 148 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:26453
[junit4:junit4] 2> 149 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0001 with negotiated timeout 10000 for client /140.211.11.196:26453
[junit4:junit4] 2> 150 T2114 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0001, negotiated timeout = 10000
[junit4:junit4] 2> 150 T2115 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3449cae name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 150 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 151 T2106 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 198 T2106 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 202 T2106 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 206 T2106 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 257 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 258 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 309 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 310 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 413 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 414 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 417 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 417 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 434 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 435 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 437 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 438 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 441 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 442 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 444 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 445 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 448 T2106 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 449 T2106 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 451 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0001
[junit4:junit4] 2> 458 T2115 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 458 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:26453 which had sessionid 0x13d4ea2f76a0001
[junit4:junit4] 2> 458 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0001 closed
[junit4:junit4] 2> 646 T2106 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 652 T2106 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24520
[junit4:junit4] 2> 653 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 654 T2106 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 654 T2106 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295
[junit4:junit4] 2> 655 T2106 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/solr.xml
[junit4:junit4] 2> 655 T2106 oasc.CoreContainer.<init> New CoreContainer 613831643
[junit4:junit4] 2> 656 T2106 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/'
[junit4:junit4] 2> 656 T2106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/'
[junit4:junit4] 2> 720 T2106 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 721 T2106 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 721 T2106 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 722 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 722 T2106 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 723 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 723 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 724 T2106 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 724 T2106 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 725 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 745 T2106 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 764 T2106 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11785/solr
[junit4:junit4] 2> 765 T2106 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 766 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@53981334
[junit4:junit4] 2> 767 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 767 T2125 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 769 T2125 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 769 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48281
[junit4:junit4] 2> 769 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48281
[junit4:junit4] 2> 775 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0002 with negotiated timeout 20000 for client /140.211.11.196:48281
[junit4:junit4] 2> 776 T2125 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0002, negotiated timeout = 20000
[junit4:junit4] 2> 776 T2126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53981334 name:ZooKeeperConnection Watcher:127.0.0.1:11785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 776 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 778 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0002
[junit4:junit4] 2> 785 T2126 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 785 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:48281 which had sessionid 0x13d4ea2f76a0002
[junit4:junit4] 2> 785 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0002 closed
[junit4:junit4] 2> 786 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 795 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4060929b
[junit4:junit4] 2> 796 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 797 T2127 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 797 T2127 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 797 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:21519
[junit4:junit4] 2> 798 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:21519
[junit4:junit4] 2> 799 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0003 with negotiated timeout 20000 for client /140.211.11.196:21519
[junit4:junit4] 2> 799 T2127 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0003, negotiated timeout = 20000
[junit4:junit4] 2> 799 T2128 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4060929b name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 799 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 802 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 810 T2106 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 812 T2106 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24520__%2Fz
[junit4:junit4] 2> 813 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:24520__%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:24520__%2Fz
[junit4:junit4] 2> 814 T2106 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24520__%2Fz
[junit4:junit4] 2> 822 T2106 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 831 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 838 T2106 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 841 T2106 oasc.Overseer.start Overseer (id=89314029459865603-127.0.0.1:24520__%2Fz-n_0000000000) starting
[junit4:junit4] 2> 841 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 843 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 850 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 851 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 858 T2130 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 859 T2106 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 862 T2106 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 873 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 875 T2129 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 879 T2131 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/collection1
[junit4:junit4] 2> 879 T2131 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 880 T2131 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 880 T2131 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 881 T2131 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/collection1/'
[junit4:junit4] 2> 882 T2131 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 883 T2131 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/collection1/lib/README' to classloader
[junit4:junit4] 2> 933 T2131 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 994 T2131 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1095 T2131 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 1101 T2131 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1750 T2131 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1761 T2131 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1765 T2131 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1778 T2131 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1782 T2131 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1787 T2131 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1788 T2131 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1788 T2131 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1788 T2131 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1789 T2131 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1790 T2131 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1790 T2131 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362823936295/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data/
[junit4:junit4] 2> 1790 T2131 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12ec5af6
[junit4:junit4] 2> 1791 T2131 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1791 T2131 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data forceNew: false
[junit4:junit4] 2> 1792 T2131 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data/index/
[junit4:junit4] 2> 1793 T2131 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1794 T2131 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data/index forceNew: false
[junit4:junit4] 2> 1831 T2131 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1832 T2131 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1834 T2131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1835 T2131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1835 T2131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1836 T2131 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1837 T2131 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1837 T2131 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1837 T2131 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1838 T2131 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1838 T2131 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1851 T2131 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1858 T2131 oass.SolrIndexSearcher.<init> Opening Searcher@681b7625 main
[junit4:junit4] 2> 1859 T2131 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/control/data/tlog
[junit4:junit4] 2> 1860 T2131 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1860 T2131 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1864 T2132 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@681b7625 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1865 T2131 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1865 T2131 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1866 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 2378 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2379 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:24520__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24520/_/z"}
[junit4:junit4] 2> 2379 T2129 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2379 T2129 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 2380 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 2401 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2882 T2131 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2882 T2131 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24520/_/z collection:control_collection shard:shard1
[junit4:junit4] 2> 2883 T2131 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2898 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2904 T2131 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2904 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2906 T2131 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2907 T2131 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2907 T2131 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24520/_/z/collection1/
[junit4:junit4] 2> 2907 T2131 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2907 T2131 oasc.SyncStrategy.syncToMe http://127.0.0.1:24520/_/z/collection1/ has no replicas
[junit4:junit4] 2> 2908 T2131 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24520/_/z/collection1/
[junit4:junit4] 2> 2908 T2131 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2912 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3905 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3921 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3934 T2131 oasc.ZkController.register We are http://127.0.0.1:24520/_/z/collection1/ and leader is http://127.0.0.1:24520/_/z/collection1/
[junit4:junit4] 2> 3934 T2131 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24520/_/z
[junit4:junit4] 2> 3934 T2131 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3934 T2131 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3935 T2131 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3936 T2131 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3937 T2106 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 3938 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3939 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3952 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3953 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7146a71c
[junit4:junit4] 2> 3954 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3955 T2134 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3955 T2134 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 3955 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31698
[junit4:junit4] 2> 3956 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31698
[junit4:junit4] 2> 3957 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0004 with negotiated timeout 10000 for client /140.211.11.196:31698
[junit4:junit4] 2> 3957 T2134 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0004, negotiated timeout = 10000
[junit4:junit4] 2> 3957 T2135 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7146a71c name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3957 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3959 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3961 T2106 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4150 T2106 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4153 T2106 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:46659
[junit4:junit4] 2> 4154 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4154 T2106 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4155 T2106 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798
[junit4:junit4] 2> 4155 T2106 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/solr.xml
[junit4:junit4] 2> 4156 T2106 oasc.CoreContainer.<init> New CoreContainer 1236521045
[junit4:junit4] 2> 4156 T2106 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/'
[junit4:junit4] 2> 4157 T2106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/'
[junit4:junit4] 2> 4220 T2106 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4221 T2106 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4222 T2106 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4222 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4223 T2106 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4223 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4224 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4224 T2106 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4225 T2106 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4225 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4244 T2106 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4287 T2106 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11785/solr
[junit4:junit4] 2> 4288 T2106 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4288 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a79e877
[junit4:junit4] 2> 4289 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4290 T2145 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4291 T2145 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 4291 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:37456
[junit4:junit4] 2> 4291 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:37456
[junit4:junit4] 2> 4297 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0005 with negotiated timeout 20000 for client /140.211.11.196:37456
[junit4:junit4] 2> 4297 T2145 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0005, negotiated timeout = 20000
[junit4:junit4] 2> 4298 T2146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a79e877 name:ZooKeeperConnection Watcher:127.0.0.1:11785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4298 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4299 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0005
[junit4:junit4] 2> 4309 T2146 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4309 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:37456 which had sessionid 0x13d4ea2f76a0005
[junit4:junit4] 2> 4309 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0005 closed
[junit4:junit4] 2> 4310 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4319 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7f1b72dd
[junit4:junit4] 2> 4320 T2147 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4320 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4321 T2147 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 4321 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17805
[junit4:junit4] 2> 4321 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17805
[junit4:junit4] 2> 4322 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0006 with negotiated timeout 20000 for client /140.211.11.196:17805
[junit4:junit4] 2> 4322 T2147 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0006, negotiated timeout = 20000
[junit4:junit4] 2> 4323 T2148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f1b72dd name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4323 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4324 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4331 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4333 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5336 T2106 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46659__%2Fz
[junit4:junit4] 2> 5337 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:46659__%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:46659__%2Fz
[junit4:junit4] 2> 5339 T2106 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46659__%2Fz
[junit4:junit4] 2> 5342 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 5342 T2135 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5343 T2148 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5344 T2128 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 5364 T2149 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/collection1
[junit4:junit4] 2> 5364 T2149 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 5365 T2149 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 5365 T2149 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 5366 T2149 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/collection1/'
[junit4:junit4] 2> 5367 T2149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 5368 T2149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/collection1/lib/README' to classloader
[junit4:junit4] 2> 5418 T2149 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 5435 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 5436 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:24520__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24520/_/z"}
[junit4:junit4] 2> 5451 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5451 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5451 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5480 T2149 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5580 T2149 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5586 T2149 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6253 T2149 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6264 T2149 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6268 T2149 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6281 T2149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6286 T2149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6290 T2149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6291 T2149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6293 T2149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6293 T2149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6294 T2149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6294 T2149 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6294 T2149 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1/
[junit4:junit4] 2> 6295 T2149 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12ec5af6
[junit4:junit4] 2> 6295 T2149 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6296 T2149 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1 forceNew: false
[junit4:junit4] 2> 6296 T2149 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1/index/
[junit4:junit4] 2> 6297 T2149 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6299 T2149 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1/index forceNew: false
[junit4:junit4] 2> 6304 T2149 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6304 T2149 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6307 T2149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6307 T2149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6308 T2149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6309 T2149 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6309 T2149 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6310 T2149 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6310 T2149 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6311 T2149 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6311 T2149 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6324 T2149 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6331 T2149 oass.SolrIndexSearcher.<init> Opening Searcher@3d2f5e7e main
[junit4:junit4] 2> 6332 T2149 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty1/tlog
[junit4:junit4] 2> 6333 T2149 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6333 T2149 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6337 T2150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d2f5e7e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6340 T2149 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 6340 T2149 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6955 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6956 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:46659__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46659/_/z"}
[junit4:junit4] 2> 6956 T2129 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6957 T2129 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6972 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6972 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6972 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7342 T2149 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7342 T2149 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46659/_/z collection:collection1 shard:shard1
[junit4:junit4] 2> 7343 T2149 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7357 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7359 T2149 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7359 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7367 T2149 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7368 T2149 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7368 T2149 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46659/_/z/collection1/
[junit4:junit4] 2> 7368 T2149 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7368 T2149 oasc.SyncStrategy.syncToMe http://127.0.0.1:46659/_/z/collection1/ has no replicas
[junit4:junit4] 2> 7369 T2149 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46659/_/z/collection1/
[junit4:junit4] 2> 7369 T2149 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7373 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8476 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8497 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8497 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8497 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8506 T2149 oasc.ZkController.register We are http://127.0.0.1:46659/_/z/collection1/ and leader is http://127.0.0.1:46659/_/z/collection1/
[junit4:junit4] 2> 8506 T2149 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46659/_/z
[junit4:junit4] 2> 8506 T2149 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8506 T2149 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8507 T2149 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8508 T2149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8509 T2106 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 8510 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8511 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8713 T2106 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8717 T2106 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25644
[junit4:junit4] 2> 8717 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8718 T2106 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8719 T2106 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360
[junit4:junit4] 2> 8719 T2106 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/solr.xml
[junit4:junit4] 2> 8719 T2106 oasc.CoreContainer.<init> New CoreContainer 475340147
[junit4:junit4] 2> 8720 T2106 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/'
[junit4:junit4] 2> 8721 T2106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/'
[junit4:junit4] 2> 8784 T2106 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8785 T2106 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8786 T2106 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8786 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8787 T2106 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8787 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8788 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8788 T2106 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8789 T2106 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8789 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8809 T2106 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8828 T2106 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11785/solr
[junit4:junit4] 2> 8829 T2106 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8830 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@16cb945b
[junit4:junit4] 2> 8831 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8831 T2161 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8832 T2161 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 8832 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16004
[junit4:junit4] 2> 8833 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16004
[junit4:junit4] 2> 8835 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0007 with negotiated timeout 20000 for client /140.211.11.196:16004
[junit4:junit4] 2> 8835 T2161 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0007, negotiated timeout = 20000
[junit4:junit4] 2> 8835 T2162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16cb945b name:ZooKeeperConnection Watcher:127.0.0.1:11785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8835 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8837 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0007
[junit4:junit4] 2> 8854 T2162 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8854 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:16004 which had sessionid 0x13d4ea2f76a0007
[junit4:junit4] 2> 8853 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0007 closed
[junit4:junit4] 2> 8855 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8864 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@42d91269
[junit4:junit4] 2> 8866 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8866 T2163 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8867 T2163 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 8867 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:26515
[junit4:junit4] 2> 8867 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:26515
[junit4:junit4] 2> 8869 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0008 with negotiated timeout 20000 for client /140.211.11.196:26515
[junit4:junit4] 2> 8869 T2163 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0008, negotiated timeout = 20000
[junit4:junit4] 2> 8869 T2164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42d91269 name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8869 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8870 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8872 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8874 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9877 T2106 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25644__%2Fz
[junit4:junit4] 2> 9878 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25644__%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25644__%2Fz
[junit4:junit4] 2> 9880 T2106 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25644__%2Fz
[junit4:junit4] 2> 9908 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9908 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9909 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9909 T2164 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9911 T2135 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9912 T2148 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9912 T2128 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9917 T2165 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/collection1
[junit4:junit4] 2> 9918 T2165 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9918 T2165 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9919 T2165 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9920 T2165 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/collection1/'
[junit4:junit4] 2> 9921 T2165 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/collection1/lib/README' to classloader
[junit4:junit4] 2> 9922 T2165 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9972 T2165 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 10002 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10003 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:46659__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46659/_/z"}
[junit4:junit4] 2> 10024 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10024 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10024 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10024 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10031 T2165 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 10049 T2165 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 10055 T2165 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 10697 T2165 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 10708 T2165 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 10711 T2165 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 10724 T2165 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10728 T2165 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10732 T2165 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10733 T2165 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10734 T2165 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10734 T2165 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10735 T2165 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10735 T2165 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10735 T2165 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362823944360/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2/
[junit4:junit4] 2> 10736 T2165 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12ec5af6
[junit4:junit4] 2> 10736 T2165 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10737 T2165 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2 forceNew: false
[junit4:junit4] 2> 10737 T2165 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2/index/
[junit4:junit4] 2> 10738 T2165 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 10739 T2165 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2/index forceNew: false
[junit4:junit4] 2> 10744 T2165 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 10745 T2165 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 10747 T2165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 10748 T2165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 10748 T2165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 10749 T2165 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 10750 T2165 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 10750 T2165 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 10750 T2165 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 10751 T2165 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 10751 T2165 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 10763 T2165 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 10770 T2165 oass.SolrIndexSearcher.<init> Opening Searcher@6f9d50d2 main
[junit4:junit4] 2> 10770 T2165 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty2/tlog
[junit4:junit4] 2> 10771 T2165 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 10771 T2165 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 10776 T2166 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f9d50d2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 10778 T2165 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 10778 T2165 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11551 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11552 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:25644__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25644/_/z"}
[junit4:junit4] 2> 11552 T2129 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11552 T2129 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11585 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11585 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11585 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11585 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11780 T2165 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11780 T2165 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25644/_/z collection:collection1 shard:shard2
[junit4:junit4] 2> 11781 T2165 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 11795 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:delete cxid:0x47 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 11796 T2165 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11797 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:create cxid:0x48 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11807 T2165 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11807 T2165 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11807 T2165 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25644/_/z/collection1/
[junit4:junit4] 2> 11808 T2165 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11808 T2165 oasc.SyncStrategy.syncToMe http://127.0.0.1:25644/_/z/collection1/ has no replicas
[junit4:junit4] 2> 11808 T2165 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25644/_/z/collection1/
[junit4:junit4] 2> 11808 T2165 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 11811 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0008 type:create cxid:0x51 zxid:0x83 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13102 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13137 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13137 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13137 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13137 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13191 T2165 oasc.ZkController.register We are http://127.0.0.1:25644/_/z/collection1/ and leader is http://127.0.0.1:25644/_/z/collection1/
[junit4:junit4] 2> 13191 T2165 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25644/_/z
[junit4:junit4] 2> 13191 T2165 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 13191 T2165 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 13192 T2165 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13193 T2165 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13194 T2106 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 13195 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 13196 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 13397 T2106 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 13401 T2106 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52185
[junit4:junit4] 2> 13401 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 13402 T2106 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 13402 T2106 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045
[junit4:junit4] 2> 13403 T2106 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/solr.xml
[junit4:junit4] 2> 13403 T2106 oasc.CoreContainer.<init> New CoreContainer 773201707
[junit4:junit4] 2> 13404 T2106 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/'
[junit4:junit4] 2> 13405 T2106 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/'
[junit4:junit4] 2> 13468 T2106 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 13469 T2106 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 13470 T2106 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 13470 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 13471 T2106 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 13471 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 13472 T2106 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 13472 T2106 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 13473 T2106 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 13473 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 13494 T2106 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 13514 T2106 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:11785/solr
[junit4:junit4] 2> 13515 T2106 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 13515 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d445c04
[junit4:junit4] 2> 13517 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13517 T2177 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13518 T2177 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 13518 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:64114
[junit4:junit4] 2> 13519 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:64114
[junit4:junit4] 2> 13521 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a0009 with negotiated timeout 20000 for client /140.211.11.196:64114
[junit4:junit4] 2> 13521 T2177 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a0009, negotiated timeout = 20000
[junit4:junit4] 2> 13521 T2178 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d445c04 name:ZooKeeperConnection Watcher:127.0.0.1:11785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13521 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13523 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a0009
[junit4:junit4] 2> 13535 T2178 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 13535 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:64114 which had sessionid 0x13d4ea2f76a0009
[junit4:junit4] 2> 13535 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a0009 closed
[junit4:junit4] 2> 13536 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 13548 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@190e1417
[junit4:junit4] 2> 13549 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 13549 T2179 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 13550 T2179 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 13550 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:60781
[junit4:junit4] 2> 13550 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:60781
[junit4:junit4] 2> 13551 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a000a with negotiated timeout 20000 for client /140.211.11.196:60781
[junit4:junit4] 2> 13551 T2179 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a000a, negotiated timeout = 20000
[junit4:junit4] 2> 13552 T2180 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@190e1417 name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 13552 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 13553 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a000a type:create cxid:0x1 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13555 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a000a type:create cxid:0x2 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 13557 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 14560 T2106 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52185__%2Fz
[junit4:junit4] 2> 14561 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a000a type:delete cxid:0xb zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:52185__%2Fz Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:52185__%2Fz
[junit4:junit4] 2> 14567 T2106 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52185__%2Fz
[junit4:junit4] 2> 14569 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14569 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14569 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14569 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14571 T2180 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14572 T2164 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14572 T2135 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14573 T2148 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14584 T2128 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 14589 T2181 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/collection1
[junit4:junit4] 2> 14590 T2181 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 14590 T2181 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 14591 T2181 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 14592 T2181 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/collection1/'
[junit4:junit4] 2> 14593 T2181 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/collection1/lib/README' to classloader
[junit4:junit4] 2> 14594 T2181 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 14643 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14644 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:25644__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25644/_/z"}
[junit4:junit4] 2> 14644 T2181 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 14656 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14656 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14657 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14657 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14656 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14703 T2181 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 14804 T2181 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 14810 T2181 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15422 T2181 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15433 T2181 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15437 T2181 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15450 T2181 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15455 T2181 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15459 T2181 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15460 T2181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15461 T2181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15461 T2181 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15462 T2181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15462 T2181 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15463 T2181 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362823949045/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/
[junit4:junit4] 2> 15463 T2181 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12ec5af6
[junit4:junit4] 2> 15464 T2181 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15464 T2181 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3 forceNew: false
[junit4:junit4] 2> 15465 T2181 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index/
[junit4:junit4] 2> 15465 T2181 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15467 T2181 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index forceNew: false
[junit4:junit4] 2> 15478 T2181 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15478 T2181 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15481 T2181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15482 T2181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15482 T2181 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15483 T2181 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15484 T2181 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15484 T2181 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15484 T2181 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15485 T2181 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15485 T2181 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15498 T2181 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15505 T2181 oass.SolrIndexSearcher.<init> Opening Searcher@7993447e main
[junit4:junit4] 2> 15506 T2181 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/tlog
[junit4:junit4] 2> 15507 T2181 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15507 T2181 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15511 T2182 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7993447e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15513 T2181 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15514 T2181 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16161 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16162 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52185__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52185/_/z"}
[junit4:junit4] 2> 16162 T2129 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 16162 T2129 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16167 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16167 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16167 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16167 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16167 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16520 T2181 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16520 T2181 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52185/_/z collection:collection1 shard:shard1
[junit4:junit4] 2> 16523 T2181 oasc.ZkController.register We are http://127.0.0.1:52185/_/z/collection1/ and leader is http://127.0.0.1:46659/_/z/collection1/
[junit4:junit4] 2> 16523 T2181 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52185/_/z
[junit4:junit4] 2> 16523 T2181 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16523 T2181 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C176 name=collection1 org.apache.solr.core.SolrCore@3d01060f url=http://127.0.0.1:52185/_/z/collection1 node=127.0.0.1:52185__%2Fz C176_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:52185__%2Fz, base_url=http://127.0.0.1:52185/_/z}
[junit4:junit4] 2> 16525 T2183 C176 P52185 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16525 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16525 T2181 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16525 T2183 C176 P52185 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16526 T2183 C176 P52185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16526 T2106 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 16527 T2183 C176 P52185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16527 T2106 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16528 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16547 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16549 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16560 T2139 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/onenodecollectioncore
[junit4:junit4] 2> 16561 T2139 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16561 T2139 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16562 T2139 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16563 T2139 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16563 T2139 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16565 T2139 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16567 T2139 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/onenodecollectioncore/'
[junit4:junit4] 2> 16637 T2139 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16697 T2139 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16798 T2139 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16804 T2139 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17434 T2139 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17445 T2139 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17449 T2139 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17462 T2139 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17467 T2139 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17471 T2139 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17472 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17474 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17474 T2139 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17475 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17475 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17476 T2139 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362823939798/onenodecollectioncore/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection/
[junit4:junit4] 2> 17476 T2139 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12ec5af6
[junit4:junit4] 2> 17477 T2139 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17478 T2139 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection forceNew: false
[junit4:junit4] 2> 17478 T2139 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection/index/
[junit4:junit4] 2> 17479 T2139 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17481 T2139 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection/index forceNew: false
[junit4:junit4] 2> 17510 T2139 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240882tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5930ba9c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17510 T2139 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17513 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17514 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17515 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17515 T2139 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17516 T2139 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17516 T2139 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17517 T2139 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17517 T2139 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17518 T2139 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17531 T2139 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17538 T2139 oass.SolrIndexSearcher.<init> Opening Searcher@26c5c581 main
[junit4:junit4] 2> 17538 T2139 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection/tlog
[junit4:junit4] 2> 17539 T2139 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17540 T2139 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17543 T2185 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@26c5c581 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17545 T2139 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 17688 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17689 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52185__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52185/_/z"}
[junit4:junit4] 2> 17692 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:46659__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46659/_/z"}
[junit4:junit4] 2> 17692 T2129 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 17693 T2129 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17709 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17709 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17709 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17709 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17709 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18549 T2139 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 18549 T2139 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:46659/_/z collection:onenodecollection shard:shard1
[junit4:junit4] 2> 18550 T2138 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=4&state=recovering&nodeName=127.0.0.1:52185__%252Fz&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 18551 T2139 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 18578 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:delete cxid:0xa1 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 18587 T2139 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 18587 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0xa2 zxid:0xae txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18595 T2139 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 18595 T2139 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 18596 T2139 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46659/_/z/onenodecollectioncore/
[junit4:junit4] 2> 18596 T2139 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 18596 T2139 oasc.SyncStrategy.syncToMe http://127.0.0.1:46659/_/z/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 18597 T2139 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46659/_/z/onenodecollectioncore/
[junit4:junit4] 2> 18597 T2139 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 18601 T2111 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d4ea2f76a0006 type:create cxid:0xac zxid:0xb2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 19215 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19237 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19237 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19237 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19237 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19237 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19276 T2139 oasc.ZkController.register We are http://127.0.0.1:46659/_/z/onenodecollectioncore/ and leader is http://127.0.0.1:46659/_/z/onenodecollectioncore/
[junit4:junit4] 2> 19276 T2139 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:46659/_/z
[junit4:junit4] 2> 19277 T2139 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 19277 T2139 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 19279 T2139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19280 T2139 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&roles=none&name=onenodecollectioncore&action=CREATE&collection=onenodecollection&dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1362823935843/onenodecollection&wt=javabin&version=2} status=0 QTime=2720
[junit4:junit4] 2> 19281 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 19282 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20284 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:52185__%2Fz, base_url=http://127.0.0.1:52185/_/z}
[junit4:junit4] 2> 20552 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:46659/_/z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20552 T2183 C176 P52185 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52185/_/z START replicas=[http://127.0.0.1:46659/_/z/collection1/] nUpdates=100
[junit4:junit4] 2> 20552 T2183 C176 P52185 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20553 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20553 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20553 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20553 T2183 C176 P52185 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20554 T2183 C176 P52185 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:46659/_/z/collection1/. core=collection1
[junit4:junit4] 2> 20554 T2183 C176 P52185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C177 name=collection1 org.apache.solr.core.SolrCore@550e1202 url=http://127.0.0.1:46659/_/z/collection1 node=127.0.0.1:46659__%2Fz C177_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:46659__%2Fz, base_url=http://127.0.0.1:46659/_/z, leader=true}
[junit4:junit4] 2> 20560 T2140 C177 P46659 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 20569 T2141 C177 P46659 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20573 T2141 C177 P46659 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20573 T2141 C177 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20586 T2141 C177 P46659 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20587 T2141 C177 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20588 T2141 C177 P46659 oass.SolrIndexSearcher.<init> Opening Searcher@43453354 realtime
[junit4:junit4] 2> 20588 T2141 C177 P46659 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20588 T2141 C177 P46659 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> 20589 T2183 C176 P52185 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20590 T2183 C176 P52185 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20593 T2142 C177 P46659 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20594 T2142 C177 P46659 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3
[junit4:junit4] 2> 20594 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 20595 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20595 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20598 T2142 C177 P46659 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 20598 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20599 T2183 C176 P52185 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index.20130309061236435 forceNew: false
[junit4:junit4] 2> 20600 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@7d45bee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d4e48e8) fullCopy=false
[junit4:junit4] 2> 20603 T2142 C177 P46659 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 20605 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20606 T2183 C176 P52185 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20606 T2183 C176 P52185 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20610 T2183 C176 P52185 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20610 T2183 C176 P52185 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20611 T2183 C176 P52185 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20611 T2183 C176 P52185 oass.SolrIndexSearcher.<init> Opening Searcher@7c582049 main
[junit4:junit4] 2> 20612 T2182 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7c582049 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 20613 T2183 C176 P52185 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@7d45bee2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d4e48e8)
[junit4:junit4] 2> 20613 T2183 C176 P52185 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20613 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20614 T2183 C176 P52185 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20614 T2183 C176 P52185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20618 T2183 C176 P52185 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20750 T2129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20751 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "node_name":"127.0.0.1:46659__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46659/_/z"}
[junit4:junit4] 2> 20754 T2129 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52185__%2Fz",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52185/_/z"}
[junit4:junit4] 2> 20773 T2128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20773 T2135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20773 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20773 T2164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20773 T2148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21286 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21287 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 21288 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C178 name=onenodecollectioncore org.apache.solr.core.SolrCore@640d45e6 url=http://127.0.0.1:46659/_/z/onenodecollectioncore node=127.0.0.1:46659__%2Fz C178_STATE=coll:onenodecollection core:onenodecollectioncore props:{shard=shard1, roles=none, state=active, core=onenodecollectioncore, collection=onenodecollection, node_name=127.0.0.1:46659__%2Fz, base_url=http://127.0.0.1:46659/_/z, leader=true}
[junit4:junit4] 2> 21316 T2143 C178 P46659 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240882tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5930ba9c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21317 T2143 C178 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21320 T2143 C178 P46659 /update {wt=javabin&version=2} {add=[1 (1429024493691469824)]} 0 7
[junit4:junit4] 2> 21324 T2143 C178 P46659 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21366 T2143 C178 P46659 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240882tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5930ba9c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240882tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5930ba9c),segFN=segments_2,generation=2,filenames=[_0.len, segments_2, _0.inf, _0.dat, _0.si, _0.pst, _0.fld]
[junit4:junit4] 2> 21366 T2143 C178 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.len, segments_2, _0.inf, _0.dat, _0.si, _0.pst, _0.fld]
[junit4:junit4] 2> 21373 T2143 C178 P46659 oass.SolrIndexSearcher.<init> Opening Searcher@7297dec0 main
[junit4:junit4] 2> 21374 T2143 C178 P46659 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21375 T2185 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@7297dec0 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21375 T2143 C178 P46659 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 21379 T2143 C178 P46659 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21381 T2106 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 21398 T2143 C178 P46659 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C179 name=collection1 org.apache.solr.core.SolrCore@2e96f4cb url=http://127.0.0.1:24520/_/z/collection1 node=127.0.0.1:24520__%2Fz C179_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:24520__%2Fz, base_url=http://127.0.0.1:24520/_/z, leader=true}
[junit4:junit4] 2> 21413 T2118 C179 P24520 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21413 T2118 C179 P24520 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21417 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[1 (1429024493792133120)]} 0 8
[junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:52185__%2Fz, base_url=http://127.0.0.1:52185/_/z}
[junit4:junit4] 2> 21450 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429024493820444672)]} 0 3
[junit4:junit4] 2> 21451 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1429024493820444672)]} 0 15
[junit4:junit4] 2> ASYNC NEW_CORE C180 name=collection1 org.apache.solr.core.SolrCore@7d35e68f url=http://127.0.0.1:25644/_/z/collection1 node=127.0.0.1:25644__%2Fz C180_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25644__%2Fz, base_url=http://127.0.0.1:25644/_/z, leader=true}
[junit4:junit4] 2> 21452 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[1]} 0 19
[junit4:junit4] 2> 21455 T2118 C179 P24520 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21493 T2118 C179 P24520 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_2,generation=2,filenames=[_0.len, segments_2, _0.inf, _0.dat, _0.si, _0.pst, _0.fld]
[junit4:junit4] 2> 21493 T2118 C179 P24520 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0.len, segments_2, _0.inf, _0.dat, _0.si, _0.pst, _0.fld]
[junit4:junit4] 2> 21501 T2118 C179 P24520 oass.SolrIndexSearcher.<init> Opening Searcher@2803fa98 main
[junit4:junit4] 2> 21502 T2118 C179 P24520 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21502 T2132 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2803fa98 main{StandardDirectoryReader(segments_2:3 _0(5.0):C1)}
[junit4:junit4] 2> 21503 T2118 C179 P24520 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 21505 T2138 C177 P46659 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21540 T2138 C177 P46659 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_3,generation=3,filenames=[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 21540 T2138 C177 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 21544 T2138 C177 P46659 oass.SolrIndexSearcher.<init> Opening Searcher@78efce57 main
[junit4:junit4] 2> 21545 T2138 C177 P46659 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21546 T2150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78efce57 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21546 T2138 C177 P46659 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:52185/_/z/collection1/, StdNode: http://127.0.0.1:25644/_/z/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 21548 T2157 C180 P25644 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21549 T2170 C176 P52185 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21552 T2157 C180 P25644 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21553 T2157 C180 P25644 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21556 T2157 C180 P25644 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 21557 T2157 C180 P25644 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 21559 T2157 C180 P25644 oass.SolrIndexSearcher.<init> Opening Searcher@7702a56d main
[junit4:junit4] 2> 21560 T2157 C180 P25644 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21560 T2166 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7702a56d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 21561 T2157 C180 P25644 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 21588 T2170 C176 P52185 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_3,generation=3,filenames=[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 21588 T2170 C176 P52185 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 21593 T2170 C176 P52185 oass.SolrIndexSearcher.<init> Opening Searcher@6a726af8 main
[junit4:junit4] 2> 21593 T2170 C176 P52185 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21594 T2182 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a726af8 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 21595 T2170 C176 P52185 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 46
[junit4:junit4] 2> 21595 T2138 C177 P46659 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 90
[junit4:junit4] 2> 21596 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21597 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21598 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 21602 T2118 C179 P24520 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=1
[junit4:junit4] 2> 21603 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7dccf660
[junit4:junit4] 2> 21604 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 21604 T2192 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 21605 T2192 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:11785, initiating session
[junit4:junit4] 2> 21605 T2108 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:59372
[junit4:junit4] 2> 21605 T2108 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:59372
[junit4:junit4] 2> 21607 T2110 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d4ea2f76a000b with negotiated timeout 10000 for client /140.211.11.196:59372
[junit4:junit4] 2> 21607 T2192 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:11785, sessionid = 0x13d4ea2f76a000b, negotiated timeout = 10000
[junit4:junit4] 2> 21607 T2193 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7dccf660 name:ZooKeeperConnection Watcher:127.0.0.1:11785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 21608 T2106 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 21609 T2106 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 21611 T2111 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d4ea2f76a000b
[junit4:junit4] 2> 21612 T2193 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21612 T2108 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:59372 which had sessionid 0x13d4ea2f76a000b
[junit4:junit4] 2> 21612 T2106 oaz.ZooKeeper.close Session: 0x13d4ea2f76a000b closed
[junit4:junit4] 2> 21614 T2141 C177 P46659 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21617 T2158 C180 P25644 REQ /select {q=*:*&distrib=false&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21622 T2172 C176 P52185 REQ /select {NOW=1362823957455&shard.url=127.0.0.1:46659/_/z/collection1/|127.0.0.1:52185/_/z/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=1 status=0 QTime=0
[junit4:junit4] 2> 21622 T2159 C180 P25644 REQ /select {NOW=1362823957455&shard.url=127.0.0.1:25644/_/z/collection1/&fl=id,score&start=0&q=*:*&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21626 T2172 C176 P52185 REQ /select {NOW=1362823957455&shard.url=127.0.0.1:46659/_/z/collection1/|127.0.0.1:52185/_/z/collection1/&ids=1&q=*:*&distrib=false&wt=javabin&isShard=true&version=2} status=0 QTime=0
[junit4:junit4] 2> 21628 T2171 C176 P52185 REQ /select {q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=9
[junit4:junit4] 2> 21635 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[2 (1429024494024916992)]} 0 4
[junit4:junit4] 2> 21645 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1429024494035402752)]} 0 4
[junit4:junit4] 2> 21645 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[2]} 0 7
[junit4:junit4] 2> 21650 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[3 (1429024494042742784)]} 0 2
[junit4:junit4] 2> 21655 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[3 (1429024494047985664)]} 0 2
[junit4:junit4] 2> 21660 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[4 (1429024494053228544)]} 0 2
[junit4:junit4] 2> 21674 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1429024494060568576)]} 0 3
[junit4:junit4] 2> 21675 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1429024494060568576)]} 0 10
[junit4:junit4] 2> 21676 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[4]} 0 14
[junit4:junit4] 2> 21681 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[5 (1429024494075248640)]} 0 2
[junit4:junit4] 2> 21686 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[5 (1429024494080491520)]} 0 2
[junit4:junit4] 2> 21691 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[6 (1429024494085734400)]} 0 2
[junit4:junit4] 2> 21698 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1429024494093074432)]} 0 2
[junit4:junit4] 2> 21698 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[6]} 0 5
[junit4:junit4] 2> 21703 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[7 (1429024494098317312)]} 0 2
[junit4:junit4] 2> 21711 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1429024494106705920)]} 0 2
[junit4:junit4] 2> 21711 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[7]} 0 5
[junit4:junit4] 2> 21716 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[8 (1429024494111948800)]} 0 2
[junit4:junit4] 2> 21725 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1429024494117191680)]} 0 2
[junit4:junit4] 2> 21725 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[8 (1429024494117191680)]} 0 6
[junit4:junit4] 2> 21730 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[9 (1429024494126628864)]} 0 2
[junit4:junit4] 2> 21737 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1429024494133968896)]} 0 2
[junit4:junit4] 2> 21737 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[9]} 0 5
[junit4:junit4] 2> 21742 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[10 (1429024494139211776)]} 0 2
[junit4:junit4] 2> 21749 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1429024494142357504)]} 0 2
[junit4:junit4] 2> 21749 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[10 (1429024494142357504)]} 0 6
[junit4:junit4] 2> 21754 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[11 (1429024494151794688)]} 0 2
[junit4:junit4] 2> 21763 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1429024494157037568)]} 0 2
[junit4:junit4] 2> 21763 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[11 (1429024494157037568)]} 0 6
[junit4:junit4] 2> 21768 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[12 (1429024494166474752)]} 0 2
[junit4:junit4] 2> 21780 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1429024494174863360)]} 0 2
[junit4:junit4] 2> 21780 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1429024494174863360)]} 0 6
[junit4:junit4] 2> 21781 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[12]} 0 10
[junit4:junit4] 2> 21786 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[13 (1429024494185349120)]} 0 2
[junit4:junit4] 2> 21798 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1429024494193737728)]} 0 2
[junit4:junit4] 2> 21798 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1429024494193737728)]} 0 6
[junit4:junit4] 2> 21799 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[13]} 0 10
[junit4:junit4] 2> 21803 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[14 (1429024494204223488)]} 0 1
[junit4:junit4] 2> 21814 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1429024494211563520)]} 0 1
[junit4:junit4] 2> 21815 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1429024494211563520)]} 0 6
[junit4:junit4] 2> 21816 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[14]} 0 10
[junit4:junit4] 2> 21820 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[15 (1429024494222049280)]} 0 1
[junit4:junit4] 2> 21828 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1429024494226243584)]} 0 1
[junit4:junit4] 2> 21829 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[15 (1429024494226243584)]} 0 6
[junit4:junit4] 2> 21833 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[16 (1429024494235680768)]} 0 1
[junit4:junit4] 2> 21844 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1429024494243020800)]} 0 1
[junit4:junit4] 2> 21845 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1429024494243020800)]} 0 6
[junit4:junit4] 2> 21846 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[16]} 0 10
[junit4:junit4] 2> 21852 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[17 (1429024494254555136)]} 0 3
[junit4:junit4] 2> 21858 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[17 (1429024494260846592)]} 0 3
[junit4:junit4] 2> 21862 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[100 (1429024494266089472)]} 0 1
[junit4:junit4] 2> 21873 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[100 (1429024494274478080)]} 0 1
[junit4:junit4] 2> 21874 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[100 (1429024494274478080)]} 0 6
[junit4:junit4] 2> 21875 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[100]} 0 10
[junit4:junit4] 2> 21879 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[101 (1429024494283915264)]} 0 1
[junit4:junit4] 2> 21890 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[101 (1429024494291255296)]} 0 1
[junit4:junit4] 2> 21891 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[101 (1429024494291255296)]} 0 6
[junit4:junit4] 2> 21892 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[101]} 0 10
[junit4:junit4] 2> 21896 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[102 (1429024494301741056)]} 0 1
[junit4:junit4] 2> 21907 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[102 (1429024494309081088)]} 0 1
[junit4:junit4] 2> 21908 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[102 (1429024494309081088)]} 0 6
[junit4:junit4] 2> 21909 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[102]} 0 10
[junit4:junit4] 2> 21913 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[103 (1429024494319566848)]} 0 1
[junit4:junit4] 2> 21918 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[103 (1429024494323761152)]} 0 2
[junit4:junit4] 2> 21922 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[104 (1429024494329004032)]} 0 1
[junit4:junit4] 2> 21929 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[104 (1429024494336344064)]} 0 1
[junit4:junit4] 2> 21930 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[104]} 0 5
[junit4:junit4] 2> 21934 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[105 (1429024494341586944)]} 0 1
[junit4:junit4] 2> 21941 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[105 (1429024494348926976)]} 0 1
[junit4:junit4] 2> 21942 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[105]} 0 5
[junit4:junit4] 2> 21946 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[106 (1429024494354169856)]} 0 1
[junit4:junit4] 2> 21950 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[106 (1429024494358364160)]} 0 1
[junit4:junit4] 2> 21954 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[107 (1429024494362558464)]} 0 1
[junit4:junit4] 2> 21965 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[107 (1429024494369898496)]} 0 1
[junit4:junit4] 2> 21966 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[107 (1429024494369898496)]} 0 6
[junit4:junit4] 2> 21967 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[107]} 0 10
[junit4:junit4] 2> 21971 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[108 (1429024494380384256)]} 0 1
[junit4:junit4] 2> 21979 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[108 (1429024494384578560)]} 0 1
[junit4:junit4] 2> 21980 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[108 (1429024494384578560)]} 0 6
[junit4:junit4] 2> 21984 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[109 (1429024494394015744)]} 0 1
[junit4:junit4] 2> 21995 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[109 (1429024494401355776)]} 0 1
[junit4:junit4] 2> 21996 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[109 (1429024494401355776)]} 0 6
[junit4:junit4] 2> 21997 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[109]} 0 10
[junit4:junit4] 2> 22001 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[110 (1429024494411841536)]} 0 1
[junit4:junit4] 2> 22009 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[110 (1429024494416035840)]} 0 1
[junit4:junit4] 2> 22010 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[110 (1429024494416035840)]} 0 6
[junit4:junit4] 2> 22014 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[111 (1429024494425473024)]} 0 1
[junit4:junit4] 2> 22025 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[111 (1429024494432813056)]} 0 1
[junit4:junit4] 2> 22026 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[111 (1429024494432813056)]} 0 6
[junit4:junit4] 2> 22027 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[111]} 0 10
[junit4:junit4] 2> 22031 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[112 (1429024494443298816)]} 0 1
[junit4:junit4] 2> 22035 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[112 (1429024494447493120)]} 0 1
[junit4:junit4] 2> 22039 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[113 (1429024494451687424)]} 0 1
[junit4:junit4] 2> 22047 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[113 (1429024494455881728)]} 0 1
[junit4:junit4] 2> 22048 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[113 (1429024494455881728)]} 0 6
[junit4:junit4] 2> 22052 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[114 (1429024494465318912)]} 0 1
[junit4:junit4] 2> 22063 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[114 (1429024494472658944)]} 0 1
[junit4:junit4] 2> 22064 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[114 (1429024494472658944)]} 0 6
[junit4:junit4] 2> 22065 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[114]} 0 10
[junit4:junit4] 2> 22069 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[115 (1429024494483144704)]} 0 1
[junit4:junit4] 2> 22073 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[115 (1429024494487339008)]} 0 1
[junit4:junit4] 2> 22077 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[116 (1429024494491533312)]} 0 1
[junit4:junit4] 2> 22081 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[116 (1429024494495727616)]} 0 1
[junit4:junit4] 2> 22085 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[117 (1429024494499921920)]} 0 1
[junit4:junit4] 2> 22090 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[117 (1429024494504116224)]} 0 2
[junit4:junit4] 2> 22094 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[118 (1429024494509359104)]} 0 1
[junit4:junit4] 2> 22103 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[118 (1429024494513553408)]} 0 2
[junit4:junit4] 2> 22104 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[118 (1429024494513553408)]} 0 7
[junit4:junit4] 2> 22108 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[119 (1429024494524039168)]} 0 1
[junit4:junit4] 2> 22115 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[119 (1429024494531379200)]} 0 1
[junit4:junit4] 2> 22116 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[119]} 0 5
[junit4:junit4] 2> 22120 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[120 (1429024494536622080)]} 0 1
[junit4:junit4] 2> 22127 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[120 (1429024494543962112)]} 0 1
[junit4:junit4] 2> 22128 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[120]} 0 5
[junit4:junit4] 2> 22133 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[121 (1429024494549204992)]} 0 2
[junit4:junit4] 2> 22137 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[121 (1429024494554447872)]} 0 1
[junit4:junit4] 2> 22142 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[122 (1429024494558642176)]} 0 2
[junit4:junit4] 2> 22154 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[122 (1429024494568079360)]} 0 1
[junit4:junit4] 2> 22154 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[122 (1429024494568079360)]} 0 6
[junit4:junit4] 2> 22155 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[122]} 0 10
[junit4:junit4] 2> 22160 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[123 (1429024494577516544)]} 0 2
[junit4:junit4] 2> 22168 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[123 (1429024494582759424)]} 0 1
[junit4:junit4] 2> 22169 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[123 (1429024494582759424)]} 0 6
[junit4:junit4] 2> 22173 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[124 (1429024494592196608)]} 0 1
[junit4:junit4] 2> 22184 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[124 (1429024494599536640)]} 0 1
[junit4:junit4] 2> 22185 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[124 (1429024494599536640)]} 0 6
[junit4:junit4] 2> 22186 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[124]} 0 10
[junit4:junit4] 2> 22190 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[125 (1429024494610022400)]} 0 1
[junit4:junit4] 2> 22197 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[125 (1429024494614216704)]} 0 1
[junit4:junit4] 2> 22198 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[125 (1429024494614216704)]} 0 5
[junit4:junit4] 2> 22202 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[126 (1429024494622605312)]} 0 1
[junit4:junit4] 2> 22206 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[126 (1429024494626799616)]} 0 1
[junit4:junit4] 2> 22210 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[127 (1429024494630993920)]} 0 1
[junit4:junit4] 2> 22217 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[127 (1429024494638333952)]} 0 1
[junit4:junit4] 2> 22218 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[127]} 0 5
[junit4:junit4] 2> 22222 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[128 (1429024494643576832)]} 0 1
[junit4:junit4] 2> 22226 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[128 (1429024494647771136)]} 0 1
[junit4:junit4] 2> 22230 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[129 (1429024494651965440)]} 0 1
[junit4:junit4] 2> 22241 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[129 (1429024494659305472)]} 0 1
[junit4:junit4] 2> 22242 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[129 (1429024494659305472)]} 0 6
[junit4:junit4] 2> 22243 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[129]} 0 10
[junit4:junit4] 2> 22247 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[130 (1429024494669791232)]} 0 1
[junit4:junit4] 2> 22251 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[130 (1429024494673985536)]} 0 1
[junit4:junit4] 2> 22255 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[131 (1429024494678179840)]} 0 1
[junit4:junit4] 2> 22262 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[131 (1429024494685519872)]} 0 1
[junit4:junit4] 2> 22263 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[131]} 0 5
[junit4:junit4] 2> 22267 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[132 (1429024494690762752)]} 0 1
[junit4:junit4] 2> 22278 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[132 (1429024494698102784)]} 0 1
[junit4:junit4] 2> 22279 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[132 (1429024494698102784)]} 0 6
[junit4:junit4] 2> 22280 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[132]} 0 10
[junit4:junit4] 2> 22284 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[133 (1429024494708588544)]} 0 1
[junit4:junit4] 2> 22293 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[133 (1429024494712782848)]} 0 2
[junit4:junit4] 2> 22294 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[133 (1429024494712782848)]} 0 7
[junit4:junit4] 2> 22298 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[134 (1429024494723268608)]} 0 1
[junit4:junit4] 2> 22302 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[134 (1429024494727462912)]} 0 1
[junit4:junit4] 2> 22306 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[135 (1429024494731657216)]} 0 1
[junit4:junit4] 2> 22317 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[135 (1429024494738997248)]} 0 1
[junit4:junit4] 2> 22318 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[135 (1429024494738997248)]} 0 6
[junit4:junit4] 2> 22319 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[135]} 0 10
[junit4:junit4] 2> 22323 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[136 (1429024494749483008)]} 0 1
[junit4:junit4] 2> 22327 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[136 (1429024494753677312)]} 0 1
[junit4:junit4] 2> 22331 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[137 (1429024494757871616)]} 0 1
[junit4:junit4] 2> 22339 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[137 (1429024494765211648)]} 0 2
[junit4:junit4] 2> 22339 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[137]} 0 5
[junit4:junit4] 2> 22343 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[138 (1429024494770454528)]} 0 1
[junit4:junit4] 2> 22350 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[138 (1429024494774648832)]} 0 1
[junit4:junit4] 2> 22351 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[138 (1429024494774648832)]} 0 5
[junit4:junit4] 2> 22355 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[139 (1429024494783037440)]} 0 1
[junit4:junit4] 2> 22359 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[139 (1429024494787231744)]} 0 1
[junit4:junit4] 2> 22363 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[140 (1429024494791426048)]} 0 1
[junit4:junit4] 2> 22370 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[140 (1429024494798766080)]} 0 1
[junit4:junit4] 2> 22371 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[140]} 0 5
[junit4:junit4] 2> 22375 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[141 (1429024494804008960)]} 0 1
[junit4:junit4] 2> 22382 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[141 (1429024494811348992)]} 0 1
[junit4:junit4] 2> 22383 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[141]} 0 5
[junit4:junit4] 2> 22387 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[142 (1429024494816591872)]} 0 1
[junit4:junit4] 2> 22391 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[142 (1429024494820786176)]} 0 1
[junit4:junit4] 2> 22395 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[143 (1429024494824980480)]} 0 1
[junit4:junit4] 2> 22402 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[143 (1429024494832320512)]} 0 1
[junit4:junit4] 2> 22403 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[143]} 0 5
[junit4:junit4] 2> 22407 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[144 (1429024494837563392)]} 0 1
[junit4:junit4] 2> 22411 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[144 (1429024494841757696)]} 0 1
[junit4:junit4] 2> 22415 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[145 (1429024494845952000)]} 0 1
[junit4:junit4] 2> 22419 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[145 (1429024494850146304)]} 0 1
[junit4:junit4] 2> 22423 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[146 (1429024494854340608)]} 0 1
[junit4:junit4] 2> 22434 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[146 (1429024494861680640)]} 0 1
[junit4:junit4] 2> 22435 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[146 (1429024494861680640)]} 0 6
[junit4:junit4] 2> 22436 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[146]} 0 10
[junit4:junit4] 2> 22440 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[147 (1429024494872166400)]} 0 1
[junit4:junit4] 2> 22447 T2157 C180 P25644 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[147 (1429024494879506432)]} 0 1
[junit4:junit4] 2> 22448 T2138 C177 P46659 /update {wt=javabin&version=2} {add=[147]} 0 5
[junit4:junit4] 2> 22452 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[148 (1429024494884749312)]} 0 1
[junit4:junit4] 2> 22464 T2170 C176 P52185 /update {distrib.from=http://127.0.0.1:46659/_/z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[148 (1429024494892089344)]} 0 2
[junit4:junit4] 2> 22464 T2144 C177 P46659 /update {distrib.from=http://127.0.0.1:25644/_/z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[148 (1429024494892089344)]} 0 6
[junit4:junit4] 2> 22465 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[148]} 0 10
[junit4:junit4] 2> 22469 T2118 C179 P24520 /update {wt=javabin&version=2} {add=[149 (1429024494902575104)]} 0 1
[junit4:junit4] 2> 22473 T2156 C180 P25644 /update {wt=javabin&version=2} {add=[149 (1429024494906769408)]} 0 1
[junit4:junit4] 2> 22474 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 22475 T2106 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22476 T2106 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 22478 T2118 C179 P24520 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22574 T2118 C179 P24520 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_2,generation=2,filenames=[_0.len, segments_2, _0.inf, _0.dat, _0.si, _0.pst, _0.fld]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240878tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d59a0d9),segFN=segments_3,generation=3,filenames=[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_3]
[junit4:junit4] 2> 22575 T2118 C179 P24520 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_3]
[junit4:junit4] 2> 22595 T2118 C179 P24520 oass.SolrIndexSearcher.<init> Opening Searcher@715f1a2f main
[junit4:junit4] 2> 22596 T2118 C179 P24520 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22596 T2132 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@715f1a2f main{StandardDirectoryReader(segments_3:5 _0(5.0):C1 _1(5.0):C66)}
[junit4:junit4] 2> 22597 T2118 C179 P24520 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 119
[junit4:junit4] 2> 22601 T2156 C180 P25644 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22655 T2156 C180 P25644 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240880tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c0cd14c),segFN=segments_3,generation=3,filenames=[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 22656 T2156 C180 P25644 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> 22669 T2156 C180 P25644 oass.SolrIndexSearcher.<init> Opening Searcher@253e23b3 main
[junit4:junit4] 2> 22670 T2156 C180 P25644 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22670 T2166 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@253e23b3 main{StandardDirectoryReader(segments_3:3 _0(5.0):C35)}
[junit4:junit4] 2> 22671 T2156 C180 P25644 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:46659/_/z/collection1/, StdNode: http://127.0.0.1:52185/_/z/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 22673 T2173 C176 P52185 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22675 T2144 C177 P46659 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22752 T2173 C176 P52185 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_3,generation=3,filenames=[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240881tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@5daaff5c),segFN=segments_4,generation=4,filenames=[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_4]
[junit4:junit4] 2> 22752 T2173 C176 P52185 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_4]
[junit4:junit4] 2> 22754 T2144 C177 P46659 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_3,generation=3,filenames=[_0.len, _0.inf, _0.dat, _0.si, _0.pst, _0.fld, segments_3]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/index4086240879tmp lockFactory=org.apache.lucene.store.NativeFSLockFactory@24ff678a),segFN=segments_4,generation=4,filenames=[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_4]
[junit4:junit4] 2> 22754 T2144 C177 P46659 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_1.fld, _1.dat, _1.inf, _0.dat, _0.pst, _1.len, _0.fld, _1.pst, _0.len, _1.si, _0.inf, _0.si, segments_4]
[junit4:junit4] 2> 22761 T2173 C176 P52185 oass.SolrIndexSearcher.<init> Opening Searcher@25e9ac52 main
[junit4:junit4] 2> 22762 T2173 C176 P52185 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22763 T2182 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25e9ac52 main{StandardDirectoryReader(segments_4:5:nrt _0(5.0):C1 _1(5.0):C31)}
[junit4:junit4] 2> 22764 T2173 C176 P52185 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 91
[junit4:junit4] 2> 22764 T2144 C177 P46659 oass.SolrIndexSearcher.<init> Opening Searcher@2e1dd206 main
[junit4:junit4] 2> 22765 T2144 C177 P46659 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22765 T2150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e1dd206 main{StandardDirectoryReader(segments_4:5:nrt _0(5.0):C1 _1(5.0):C31)}
[junit4:junit4] 2> 22766 T2144 C177 P46659 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 91
[junit4:junit4] 2> 22767 T2156 C180 P25644 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 166
[junit4:junit4] 2> 22771 T2118 C179 P24520 REQ /select {q=*:*&wt=javabin&version=2} hits=67 status=0 QTime=1
[junit4:junit4] 2> 22775 T2106 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:11785/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@27c79f91
[junit4:junit4] 2> 22776 T2106 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 22776 T2197 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:11785. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "g
[...truncated too long message...]
rCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3d01060f
[junit4:junit4] 2> 88776 T2180 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=36,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 88778 T2180 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 88778 T2180 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 88778 T2180 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 88784 T2180 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 88784 T2180 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index.20130309061236435
[junit4:junit4] 2> 88784 T2180 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index.20130309061236435
[junit4:junit4] 2> 88785 T2180 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3
[junit4:junit4] 2> 88785 T2180 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3
[junit4:junit4] 2> 88785 T2180 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index
[junit4:junit4] 2> 88785 T2180 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1362823935843/jetty3/index
[junit4:junit4] 2> 88786 T2180 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 88786 T2180 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election/89314029459865610-4-n_0000000001
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:153)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:150)
[junit4:junit4] 2> at org.apache.solr.cloud.ElectionContext.cancelElection(ElectionContext.java:63)
[junit4:junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:243)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:156)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:100)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 88787 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 88787 T2180 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 88787 T2180 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 88787 T2180 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 88787 T2180 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 88788 T2180 oasc.LeaderElector$1.process WARNING org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 88788 T2180 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=pt_BR, timezone=America/Dominica
[junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=3,free=398451392,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestElisionMultitermQuery, TestSuggestSpellingConverter, PingRequestHandlerTest, NumericFieldsTest, TestNumberUtils, DocValuesMultiTest, TestJmxMonitoredMap, TestStressLucene, NoCacheHeaderTest, JSONWriterTest, TestDynamicFieldResource, SampleTest, TestWordDelimiterFilterFactory, TestHashPartitioner, TestFiltering, TestDFRSimilarityFactory, TestLuceneMatchVersion, TestDistributedGrouping, TestDynamicFieldCollectionResource, OpenExchangeRatesOrgProviderTest, TestFastOutputStream, TestIndexSearcher, TestMultiCoreConfBootstrap, SolrCoreCheckLockOnStartupTest, SignatureUpdateProcessorFactoryTest, PathHierarchyTokenizerFactoryTest, TestReload, TestQuerySenderNoQuery, TestPartialUpdateDeduplication, DateMathParserTest, BasicZkTest, TestStressReorder, TestRecovery, CurrencyFieldOpenExchangeTest, LeaderElectionIntegrationTest, TestRTGBase, TestZkChroot, ShardRoutingCustomTest, RequiredFieldsTest, DistanceFunctionTest, ZkSolrClientTest, NotRequiredUniqueKeyTest, BadIndexSchemaTest, SortByFunctionTest, TestSolrJ, TestIndexingPerformance, TestCSVLoader, SolrRequestParserTest, TestComponentsName, TestCSVResponseWriter, EchoParamsTest, SuggesterWFSTTest, AliasIntegrationTest, DirectUpdateHandlerTest, CoreContainerCoreInitFailuresTest, FullSolrCloudDistribCmdsTest, CollectionsAPIDistributedZkTest, TestLMJelinekMercerSimilarityFactory, DocumentAnalysisRequestHandlerTest, FieldAnalysisRequestHandlerTest, TestLazyCores, SolrCmdDistributorTest, TestBinaryField, UniqFieldsUpdateProcessorFactoryTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 88.96s, 1 test, 1 error <<< FAILURES!
[...truncated 454 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:449: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1213: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:877: There were test failures: 269 suites, 1137 tests, 1 error, 27 ignored (8 assumptions)
Total time: 48 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure