You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/03/16 16:33:05 UTC
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.6.0_41) - Build # 4701
- Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4701/
Java: 64bit/jdk1.6.0_41 -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([E90AE1C3E6AD6A5D:68EC6FDB91F20A61]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:196)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:662)
Build Log:
[...truncated 8660 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 0 T1880 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 4 T1880 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1363447045305
[junit4:junit4] 2> 4 T1880 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 5 T1881 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 5 T1881 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 5 T1881 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 6 T1881 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 6 T1881 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 6 T1881 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1363447045305/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 105 T1880 oasc.ZkTestServer.run start zk server on port:39998
[junit4:junit4] 2> 105 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3fa271ed
[junit4:junit4] 2> 106 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 106 T1886 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 106 T1886 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 107 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46774
[junit4:junit4] 2> 107 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46774
[junit4:junit4] 2> 107 T1884 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 109 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0000 with negotiated timeout 10000 for client /127.0.0.1:46774
[junit4:junit4] 2> 109 T1886 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0000, negotiated timeout = 10000
[junit4:junit4] 2> 110 T1887 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3fa271ed name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 110 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 110 T1880 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 112 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0000
[junit4:junit4] 2> 113 T1887 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 113 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0000 closed
[junit4:junit4] 2> 113 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46774 which had sessionid 0x13d73c6dcbc0000
[junit4:junit4] 2> 113 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6f382eae
[junit4:junit4] 2> 114 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 114 T1888 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 114 T1888 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 114 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46775
[junit4:junit4] 2> 115 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46775
[junit4:junit4] 2> 115 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0001 with negotiated timeout 10000 for client /127.0.0.1:46775
[junit4:junit4] 2> 115 T1888 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0001, negotiated timeout = 10000
[junit4:junit4] 2> 116 T1889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f382eae name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 116 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 116 T1880 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 119 T1880 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 121 T1880 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 123 T1880 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 124 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 125 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 127 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 128 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 130 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 131 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 132 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 133 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 134 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 135 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 137 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 137 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 139 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 139 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 141 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 141 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 143 T1880 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 143 T1880 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 145 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0001
[junit4:junit4] 2> 145 T1889 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 145 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0001 closed
[junit4:junit4] 2> 145 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46775 which had sessionid 0x13d73c6dcbc0001
[junit4:junit4] 2> 209 T1880 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 227 T1880 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37061
[junit4:junit4] 2> 227 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 228 T1880 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 228 T1880 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447
[junit4:junit4] 2> 228 T1880 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/solr.xml
[junit4:junit4] 2> 229 T1880 oasc.CoreContainer.<init> New CoreContainer 1688344391
[junit4:junit4] 2> 229 T1880 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/'
[junit4:junit4] 2> 229 T1880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/'
[junit4:junit4] 2> 248 T1880 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 249 T1880 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 250 T1880 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 250 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 251 T1880 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 251 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 252 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 253 T1880 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 253 T1880 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 254 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 259 T1880 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 277 T1880 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:39998/solr
[junit4:junit4] 2> 277 T1880 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 278 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5509eef7
[junit4:junit4] 2> 278 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 279 T1899 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 279 T1899 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 280 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46779
[junit4:junit4] 2> 280 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46779
[junit4:junit4] 2> 281 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0002 with negotiated timeout 20000 for client /127.0.0.1:46779
[junit4:junit4] 2> 281 T1899 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0002, negotiated timeout = 20000
[junit4:junit4] 2> 281 T1900 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5509eef7 name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 281 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 282 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0002
[junit4:junit4] 2> 283 T1900 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 283 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46779 which had sessionid 0x13d73c6dcbc0002
[junit4:junit4] 2> 283 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0002 closed
[junit4:junit4] 2> 283 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 285 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@10a11c83
[junit4:junit4] 2> 286 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 286 T1901 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 287 T1901 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 287 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46780
[junit4:junit4] 2> 287 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46780
[junit4:junit4] 2> 288 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0003 with negotiated timeout 20000 for client /127.0.0.1:46780
[junit4:junit4] 2> 288 T1901 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0003, negotiated timeout = 20000
[junit4:junit4] 2> 289 T1902 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10a11c83 name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 289 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 290 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 292 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 293 T1880 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37061_
[junit4:junit4] 2> 294 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37061_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37061_
[junit4:junit4] 2> 294 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37061_
[junit4:junit4] 2> 297 T1880 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 300 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 300 T1880 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 303 T1880 oasc.Overseer.start Overseer (id=89354865561305091-127.0.0.1:37061_-n_0000000000) starting
[junit4:junit4] 2> 303 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 304 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 304 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 305 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 306 T1904 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 306 T1880 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 308 T1880 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 309 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 311 T1903 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 313 T1905 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/collection1
[junit4:junit4] 2> 313 T1905 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 314 T1905 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 314 T1905 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 314 T1905 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/collection1/'
[junit4:junit4] 2> 315 T1905 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/collection1/lib/README' to classloader
[junit4:junit4] 2> 315 T1905 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 341 T1905 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 370 T1905 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 371 T1905 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 374 T1905 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 700 T1905 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 705 T1905 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 708 T1905 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 717 T1905 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 720 T1905 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 722 T1905 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 723 T1905 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 723 T1905 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 723 T1905 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 724 T1905 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 724 T1905 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 724 T1905 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 725 T1905 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1363447045447/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/
[junit4:junit4] 2> 725 T1905 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@464d41e2
[junit4:junit4] 2> 725 T1905 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 725 T1905 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data forceNew: false
[junit4:junit4] 2> 726 T1905 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index/
[junit4:junit4] 2> 726 T1905 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 726 T1905 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index forceNew: false
[junit4:junit4] 2> 729 T1905 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 729 T1905 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 730 T1905 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 731 T1905 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 731 T1905 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 731 T1905 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 732 T1905 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 732 T1905 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 732 T1905 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 732 T1905 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 733 T1905 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 734 T1905 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 737 T1905 oass.SolrIndexSearcher.<init> Opening Searcher@1333e6e6 main
[junit4:junit4] 2> 737 T1905 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 737 T1905 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 741 T1906 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1333e6e6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 741 T1905 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 742 T1905 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 742 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 1813 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1814 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:37061_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37061"}
[junit4:junit4] 2> 1814 T1903 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1814 T1903 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1814 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 1818 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2746 T1905 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2746 T1905 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37061 collection:control_collection shard:shard1
[junit4:junit4] 2> 2747 T1905 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2752 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2752 T1905 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2753 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2757 T1905 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2757 T1905 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2757 T1905 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37061/collection1/
[junit4:junit4] 2> 2757 T1905 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2758 T1905 oasc.SyncStrategy.syncToMe http://127.0.0.1:37061/collection1/ has no replicas
[junit4:junit4] 2> 2758 T1905 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37061/collection1/
[junit4:junit4] 2> 2758 T1905 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2761 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3320 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3328 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3364 T1905 oasc.ZkController.register We are http://127.0.0.1:37061/collection1/ and leader is http://127.0.0.1:37061/collection1/
[junit4:junit4] 2> 3364 T1905 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37061
[junit4:junit4] 2> 3364 T1905 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3365 T1905 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3365 T1905 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3366 T1905 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3368 T1880 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3368 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3369 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3372 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3373 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@b5702f4
[junit4:junit4] 2> 3374 T1908 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 3375 T1908 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 3375 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46817
[junit4:junit4] 2> 3375 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46817
[junit4:junit4] 2> 3376 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3376 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0004 with negotiated timeout 10000 for client /127.0.0.1:46817
[junit4:junit4] 2> 3376 T1908 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0004, negotiated timeout = 10000
[junit4:junit4] 2> 3377 T1909 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b5702f4 name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3377 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3378 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3380 T1880 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3461 T1880 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3465 T1880 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35051
[junit4:junit4] 2> 3465 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3466 T1880 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3466 T1880 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682
[junit4:junit4] 2> 3467 T1880 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/solr.xml
[junit4:junit4] 2> 3467 T1880 oasc.CoreContainer.<init> New CoreContainer 1059342631
[junit4:junit4] 2> 3468 T1880 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/'
[junit4:junit4] 2> 3468 T1880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/'
[junit4:junit4] 2> 3487 T1880 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3487 T1880 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3487 T1880 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3488 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3488 T1880 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3488 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3488 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3489 T1880 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3489 T1880 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3489 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3496 T1880 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3511 T1880 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:39998/solr
[junit4:junit4] 2> 3512 T1880 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3512 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@48a3ea99
[junit4:junit4] 2> 3513 T1919 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 3514 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3518 T1919 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 3518 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46818
[junit4:junit4] 2> 3518 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46818
[junit4:junit4] 2> 3519 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0005 with negotiated timeout 20000 for client /127.0.0.1:46818
[junit4:junit4] 2> 3519 T1919 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0005, negotiated timeout = 20000
[junit4:junit4] 2> 3520 T1920 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48a3ea99 name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3520 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3522 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0005
[junit4:junit4] 2> 3523 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46818 which had sessionid 0x13d73c6dcbc0005
[junit4:junit4] 2> 3523 T1920 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3523 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0005 closed
[junit4:junit4] 2> 3524 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3527 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@10149d92
[junit4:junit4] 2> 3528 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3528 T1921 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 3529 T1921 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 3529 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46819
[junit4:junit4] 2> 3529 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46819
[junit4:junit4] 2> 3530 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0006 with negotiated timeout 20000 for client /127.0.0.1:46819
[junit4:junit4] 2> 3530 T1921 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0006, negotiated timeout = 20000
[junit4:junit4] 2> 3530 T1922 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10149d92 name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3530 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3531 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3532 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3534 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4537 T1880 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35051_
[junit4:junit4] 2> 4538 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35051_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35051_
[junit4:junit4] 2> 4539 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35051_
[junit4:junit4] 2> 4541 T1909 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4542 T1902 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4542 T1922 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4542 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4546 T1923 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/collection1
[junit4:junit4] 2> 4546 T1923 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4547 T1923 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4547 T1923 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4547 T1923 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/collection1/'
[junit4:junit4] 2> 4548 T1923 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/collection1/lib/README' to classloader
[junit4:junit4] 2> 4548 T1923 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4571 T1923 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 4600 T1923 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4601 T1923 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4604 T1923 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4834 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4835 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "node_name":"127.0.0.1:37061_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37061"}
[junit4:junit4] 2> 4837 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4837 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4838 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4927 T1923 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 4932 T1923 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4935 T1923 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4944 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4947 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4949 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 4950 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 4951 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 4951 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 4951 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 4952 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 4952 T1923 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 4952 T1923 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1363447048682/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/
[junit4:junit4] 2> 4952 T1923 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@464d41e2
[junit4:junit4] 2> 4953 T1923 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 4953 T1923 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1 forceNew: false
[junit4:junit4] 2> 4953 T1923 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index/
[junit4:junit4] 2> 4953 T1923 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4954 T1923 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index forceNew: false
[junit4:junit4] 2> 4957 T1923 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 4958 T1923 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 4959 T1923 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4960 T1923 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4960 T1923 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4961 T1923 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4961 T1923 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4961 T1923 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4961 T1923 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4962 T1923 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4962 T1923 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4964 T1923 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4967 T1923 oass.SolrIndexSearcher.<init> Opening Searcher@2f45de16 main
[junit4:junit4] 2> 4968 T1923 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4968 T1923 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4973 T1924 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f45de16 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4976 T1923 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4976 T1923 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6342 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6343 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35051_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35051"}
[junit4:junit4] 2> 6343 T1903 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 6343 T1903 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6349 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6349 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6349 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6978 T1923 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 6978 T1923 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35051 collection:collection1 shard:shard1
[junit4:junit4] 2> 6979 T1923 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 6984 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 6985 T1923 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 6985 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 6986 T1923 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 6986 T1923 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 6986 T1923 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 6986 T1923 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 6987 T1923 oasc.SyncStrategy.syncToMe http://127.0.0.1:35051/collection1/ has no replicas
[junit4:junit4] 2> 6987 T1923 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 6987 T1923 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 6989 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7856 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7866 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7866 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7866 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7893 T1923 oasc.ZkController.register We are http://127.0.0.1:35051/collection1/ and leader is http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 7893 T1923 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35051
[junit4:junit4] 2> 7893 T1923 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 7894 T1923 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 7894 T1923 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 7896 T1923 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7898 T1880 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 7898 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 7899 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 7973 T1880 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 7975 T1880 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57820
[junit4:junit4] 2> 7976 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 7977 T1880 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 7977 T1880 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205
[junit4:junit4] 2> 7977 T1880 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/solr.xml
[junit4:junit4] 2> 7977 T1880 oasc.CoreContainer.<init> New CoreContainer 801285503
[junit4:junit4] 2> 7978 T1880 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/'
[junit4:junit4] 2> 7978 T1880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/'
[junit4:junit4] 2> 7997 T1880 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 7998 T1880 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 7998 T1880 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 7998 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 7998 T1880 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 7999 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 7999 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 7999 T1880 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8000 T1880 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8000 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8005 T1880 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8014 T1880 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:39998/solr
[junit4:junit4] 2> 8015 T1880 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8015 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e2ed6a9
[junit4:junit4] 2> 8016 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8016 T1935 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 8017 T1935 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 8017 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46827
[junit4:junit4] 2> 8018 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46827
[junit4:junit4] 2> 8019 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0007 with negotiated timeout 20000 for client /127.0.0.1:46827
[junit4:junit4] 2> 8019 T1935 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0007, negotiated timeout = 20000
[junit4:junit4] 2> 8019 T1936 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e2ed6a9 name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8020 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8021 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0007
[junit4:junit4] 2> 8022 T1936 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8022 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46827 which had sessionid 0x13d73c6dcbc0007
[junit4:junit4] 2> 8022 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0007 closed
[junit4:junit4] 2> 8023 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8025 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@105d8bde
[junit4:junit4] 2> 8025 T1937 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 8025 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8026 T1937 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 8026 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46828
[junit4:junit4] 2> 8026 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46828
[junit4:junit4] 2> 8027 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0008 with negotiated timeout 20000 for client /127.0.0.1:46828
[junit4:junit4] 2> 8027 T1937 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0008, negotiated timeout = 20000
[junit4:junit4] 2> 8028 T1938 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@105d8bde name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8028 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8029 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8030 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8033 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9036 T1880 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57820_
[junit4:junit4] 2> 9037 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc0008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57820_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57820_
[junit4:junit4] 2> 9038 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57820_
[junit4:junit4] 2> 9040 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9040 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9040 T1902 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1938 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9041 T1922 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9041 T1909 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9045 T1939 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/collection1
[junit4:junit4] 2> 9045 T1939 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9046 T1939 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9046 T1939 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9047 T1939 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/collection1/'
[junit4:junit4] 2> 9047 T1939 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/collection1/lib/README' to classloader
[junit4:junit4] 2> 9047 T1939 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9071 T1939 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 9100 T1939 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9101 T1939 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9105 T1939 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9372 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9373 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35051_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35051"}
[junit4:junit4] 2> 9376 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9376 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9376 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9376 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9434 T1939 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9439 T1939 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9442 T1939 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9455 T1939 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9458 T1939 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9462 T1939 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9463 T1939 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9464 T1939 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9464 T1939 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9465 T1939 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9465 T1939 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9465 T1939 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 9465 T1939 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1363447053205/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/
[junit4:junit4] 2> 9466 T1939 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@464d41e2
[junit4:junit4] 2> 9466 T1939 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 9466 T1939 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2 forceNew: false
[junit4:junit4] 2> 9467 T1939 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index/
[junit4:junit4] 2> 9467 T1939 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9467 T1939 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index forceNew: false
[junit4:junit4] 2> 9470 T1939 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9470 T1939 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9471 T1939 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9472 T1939 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9472 T1939 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9473 T1939 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9473 T1939 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9473 T1939 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9473 T1939 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9474 T1939 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9474 T1939 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9476 T1939 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9479 T1939 oass.SolrIndexSearcher.<init> Opening Searcher@5dc82dae main
[junit4:junit4] 2> 9479 T1939 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9479 T1939 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9483 T1940 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dc82dae main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9485 T1939 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9485 T1939 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 10880 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10881 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:57820_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57820"}
[junit4:junit4] 2> 10882 T1903 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 10882 T1903 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 10888 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10888 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10888 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10888 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11486 T1939 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11487 T1939 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57820 collection:collection1 shard:shard1
[junit4:junit4] 2> 11489 T1939 oasc.ZkController.register We are http://127.0.0.1:57820/collection1/ and leader is http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 11489 T1939 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57820
[junit4:junit4] 2> 11489 T1939 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 11489 T1939 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C128 name=collection1 org.apache.solr.core.SolrCore@714fa4b7 url=http://127.0.0.1:57820/collection1 node=127.0.0.1:57820_ C128_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57820_, base_url=http://127.0.0.1:57820}
[junit4:junit4] 2> 11490 T1941 C128 P57820 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 11490 T1939 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11490 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 11491 T1941 C128 P57820 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 11491 T1941 C128 P57820 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11491 T1880 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 11491 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 11492 T1941 C128 P57820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11492 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11553 T1880 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 11555 T1880 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35237
[junit4:junit4] 2> 11555 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 11555 T1880 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 11556 T1880 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795
[junit4:junit4] 2> 11556 T1880 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/solr.xml
[junit4:junit4] 2> 11556 T1880 oasc.CoreContainer.<init> New CoreContainer 1455375574
[junit4:junit4] 2> 11556 T1880 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/'
[junit4:junit4] 2> 11557 T1880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/'
[junit4:junit4] 2> 11573 T1880 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 11574 T1880 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 11574 T1880 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 11575 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 11575 T1880 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 11576 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 11576 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 11577 T1880 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 11577 T1880 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 11578 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 11586 T1880 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 11595 T1880 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:39998/solr
[junit4:junit4] 2> 11595 T1880 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 11596 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@75f561c9
[junit4:junit4] 2> 11596 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 11597 T1952 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 11597 T1952 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 11598 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46838
[junit4:junit4] 2> 11598 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46838
[junit4:junit4] 2> 11599 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc0009 with negotiated timeout 20000 for client /127.0.0.1:46838
[junit4:junit4] 2> 11599 T1952 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc0009, negotiated timeout = 20000
[junit4:junit4] 2> 11599 T1953 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75f561c9 name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 11599 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 11600 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc0009
[junit4:junit4] 2> 11601 T1953 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 11601 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46838 which had sessionid 0x13d73c6dcbc0009
[junit4:junit4] 2> 11601 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc0009 closed
[junit4:junit4] 2> 11602 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 11604 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@651b9647
[junit4:junit4] 2> 11605 T1954 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 11605 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 11605 T1954 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 11605 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46839
[junit4:junit4] 2> 11605 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46839
[junit4:junit4] 2> 11606 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc000a with negotiated timeout 20000 for client /127.0.0.1:46839
[junit4:junit4] 2> 11606 T1954 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc000a, negotiated timeout = 20000
[junit4:junit4] 2> 11606 T1955 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@651b9647 name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 11606 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 11607 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000a type:create cxid:0x1 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11608 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000a type:create cxid:0x2 zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11609 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 12394 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12395 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:57820_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57820"}
[junit4:junit4] 2> 12401 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12401 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12402 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12402 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12402 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12495 T1918 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=3&state=recovering&nodeName=127.0.0.1:57820_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1000
[junit4:junit4] 2> 12612 T1880 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35237_
[junit4:junit4] 2> 12613 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000a type:delete cxid:0xd zxid:0x87 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35237_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35237_
[junit4:junit4] 2> 12613 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35237_
[junit4:junit4] 2> 12615 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12615 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12615 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12616 T1938 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 12616 T1902 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 12616 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 12616 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 12617 T1922 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 12617 T1955 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 12617 T1909 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 12621 T1956 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/collection1
[junit4:junit4] 2> 12621 T1956 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 12622 T1956 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 12622 T1956 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 12622 T1956 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/collection1/'
[junit4:junit4] 2> 12623 T1956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/collection1/lib/README' to classloader
[junit4:junit4] 2> 12623 T1956 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 12651 T1956 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 12680 T1956 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 12682 T1956 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 12685 T1956 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 13021 T1956 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 13026 T1956 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 13028 T1956 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 13040 T1956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 13043 T1956 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 13045 T1956 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 13046 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 13047 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 13047 T1956 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 13048 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 13048 T1956 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 13048 T1956 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 13048 T1956 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1363447056795/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/
[junit4:junit4] 2> 13048 T1956 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@464d41e2
[junit4:junit4] 2> 13049 T1956 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 13049 T1956 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3 forceNew: false
[junit4:junit4] 2> 13049 T1956 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index/
[junit4:junit4] 2> 13049 T1956 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 13050 T1956 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index forceNew: false
[junit4:junit4] 2> 13053 T1956 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 13054 T1956 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 13055 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 13055 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 13056 T1956 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 13056 T1956 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 13057 T1956 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 13057 T1956 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 13057 T1956 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 13057 T1956 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 13058 T1956 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 13059 T1956 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 13062 T1956 oass.SolrIndexSearcher.<init> Opening Searcher@55a8d188 main
[junit4:junit4] 2> 13062 T1956 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 13062 T1956 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 13067 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55a8d188 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 13068 T1956 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 13068 T1956 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 13908 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13908 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35237_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35237"}
[junit4:junit4] 2> 13909 T1903 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 13909 T1903 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 13912 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13912 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13912 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13912 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13912 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14070 T1956 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 14070 T1956 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35237 collection:collection1 shard:shard1
[junit4:junit4] 2> 14072 T1956 oasc.ZkController.register We are http://127.0.0.1:35237/collection1/ and leader is http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 14072 T1956 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35237
[junit4:junit4] 2> 14072 T1956 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 14073 T1956 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C129 name=collection1 org.apache.solr.core.SolrCore@28e71517 url=http://127.0.0.1:35237/collection1 node=127.0.0.1:35237_ C129_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:35237_, base_url=http://127.0.0.1:35237}
[junit4:junit4] 2> 14073 T1958 C129 P35237 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 14073 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 14073 T1956 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14074 T1958 C129 P35237 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 14074 T1958 C129 P35237 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 14075 T1958 C129 P35237 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 14074 T1880 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 14075 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 14075 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 14153 T1880 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 14156 T1880 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46674
[junit4:junit4] 2> 14156 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 14156 T1880 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 14157 T1880 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379
[junit4:junit4] 2> 14157 T1880 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/solr.xml
[junit4:junit4] 2> 14157 T1880 oasc.CoreContainer.<init> New CoreContainer 170814137
[junit4:junit4] 2> 14158 T1880 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/'
[junit4:junit4] 2> 14158 T1880 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/'
[junit4:junit4] 2> 14174 T1880 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 14175 T1880 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 14175 T1880 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 14175 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 14175 T1880 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 14176 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 14176 T1880 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 14176 T1880 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 14176 T1880 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 14177 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 14182 T1880 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 14192 T1880 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:39998/solr
[junit4:junit4] 2> 14192 T1880 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 14193 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@61dd39ca
[junit4:junit4] 2> 14193 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 14194 T1969 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 14194 T1969 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 14194 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46845
[junit4:junit4] 2> 14195 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46845
[junit4:junit4] 2> 14195 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc000b with negotiated timeout 20000 for client /127.0.0.1:46845
[junit4:junit4] 2> 14195 T1969 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc000b, negotiated timeout = 20000
[junit4:junit4] 2> 14196 T1970 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61dd39ca name:ZooKeeperConnection Watcher:127.0.0.1:39998 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 14196 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 14197 T1885 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d73c6dcbc000b
[junit4:junit4] 2> 14198 T1970 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 14198 T1882 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:46845 which had sessionid 0x13d73c6dcbc000b
[junit4:junit4] 2> 14198 T1880 oaz.ZooKeeper.close Session: 0x13d73c6dcbc000b closed
[junit4:junit4] 2> 14198 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 14200 T1880 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:39998/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6c02b3e
[junit4:junit4] 2> 14201 T1880 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 14201 T1971 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:39998. Will not attempt to authenticate using SASL (access denied (javax.security.auth.AuthPermission getLoginConfiguration))
[junit4:junit4] 2> 14202 T1971 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:39998, initiating session
[junit4:junit4] 2> 14202 T1882 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:46846
[junit4:junit4] 2> 14202 T1882 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:46846
[junit4:junit4] 2> 14203 T1884 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d73c6dcbc000c with negotiated timeout 20000 for client /127.0.0.1:46846
[junit4:junit4] 2> 14203 T1971 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:39998, sessionid = 0x13d73c6dcbc000c, negotiated timeout = 20000
[junit4:junit4] 2> 14203 T1972 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c02b3e name:ZooKeeperConnection Watcher:127.0.0.1:39998/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 14203 T1880 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 14204 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000c type:create cxid:0x1 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 14205 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000c type:create cxid:0x2 zxid:0x95 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 14206 T1880 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> C128_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57820_, base_url=http://127.0.0.1:57820}
[junit4:junit4] 2> 14496 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35051/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 14496 T1941 C128 P57820 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57820 START replicas=[http://127.0.0.1:35051/collection1/] nUpdates=100
[junit4:junit4] 2> 14497 T1941 C128 P57820 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 14497 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 14497 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 14497 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 14497 T1941 C128 P57820 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 14498 T1941 C128 P57820 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35051/collection1/. core=collection1
[junit4:junit4] 2> 14498 T1941 C128 P57820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C130 name=collection1 org.apache.solr.core.SolrCore@5251a0e1 url=http://127.0.0.1:35051/collection1 node=127.0.0.1:35051_ C130_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35051_, base_url=http://127.0.0.1:35051, leader=true}
[junit4:junit4] 2> 14501 T1915 C130 P35051 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 14503 T1918 C130 P35051 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 14505 T1918 C130 P35051 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14506 T1918 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14514 T1918 C130 P35051 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 14514 T1918 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 14515 T1918 C130 P35051 oass.SolrIndexSearcher.<init> Opening Searcher@510e44a2 realtime
[junit4:junit4] 2> 14515 T1918 C130 P35051 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 14515 T1918 C130 P35051 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 14516 T1941 C128 P57820 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14516 T1941 C128 P57820 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 14535 T1915 C130 P35051 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14536 T1915 C130 P35051 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=19
[junit4:junit4] 2> 14536 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 14536 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 14536 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 14538 T1918 C130 P35051 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 14539 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 14540 T1941 C128 P57820 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index.20130316121739840 forceNew: false
[junit4:junit4] 2> 14540 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index.20130316121739840 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e64c934; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 14543 T1916 C130 P35051 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 14544 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 14555 T1941 C128 P57820 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 14555 T1941 C128 P57820 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 14556 T1941 C128 P57820 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 14557 T1941 C128 P57820 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 14557 T1941 C128 P57820 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 14557 T1941 C128 P57820 oass.SolrIndexSearcher.<init> Opening Searcher@3ff47c8a main
[junit4:junit4] 2> 14558 T1940 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ff47c8a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 14559 T1941 C128 P57820 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index.20130316121739840 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e64c934; maxCacheMB=48.0 maxMergeSizeMB=4.0)
[junit4:junit4] 2> 14559 T1941 C128 P57820 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 14559 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 14559 T1941 C128 P57820 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 14560 T1941 C128 P57820 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 14562 T1941 C128 P57820 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 15208 T1880 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46674_
[junit4:junit4] 2> 15209 T1885 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d73c6dcbc000c type:delete cxid:0xb zxid:0x97 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:46674_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:46674_
[junit4:junit4] 2> 15211 T1880 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46674_
[junit4:junit4] 2> 15213 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15213 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15213 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15214 T1938 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15215 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15215 T1902 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15215 T1972 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15215 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15216 T1909 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15216 T1922 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15217 T1955 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 15222 T1975 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/collection1
[junit4:junit4] 2> 15222 T1975 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 15223 T1975 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 15224 T1975 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 15225 T1975 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/collection1/'
[junit4:junit4] 2> 15226 T1975 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/collection1/lib/README' to classloader
[junit4:junit4] 2> 15226 T1975 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 15264 T1975 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 15294 T1975 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 15295 T1975 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 15299 T1975 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15415 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15416 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35237_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35237"}
[junit4:junit4] 2> 15418 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:57820_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57820"}
[junit4:junit4] 2> 15434 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15434 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15434 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15434 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15435 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15435 T1972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15621 T1975 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15626 T1975 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15628 T1975 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15638 T1975 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15640 T1975 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15643 T1975 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15643 T1975 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15644 T1975 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15644 T1975 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15644 T1975 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15645 T1975 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15645 T1975 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 15645 T1975 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1363447059379/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/
[junit4:junit4] 2> 15645 T1975 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@464d41e2
[junit4:junit4] 2> 15646 T1975 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 15646 T1975 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4 forceNew: false
[junit4:junit4] 2> 15646 T1975 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index/
[junit4:junit4] 2> 15646 T1975 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15647 T1975 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index forceNew: false
[junit4:junit4] 2> 15650 T1975 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19119e91; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15650 T1975 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15651 T1975 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15652 T1975 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15652 T1975 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15652 T1975 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15653 T1975 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15653 T1975 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15653 T1975 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15653 T1975 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15654 T1975 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15655 T1975 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15658 T1975 oass.SolrIndexSearcher.<init> Opening Searcher@31450cca main
[junit4:junit4] 2> 15658 T1975 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15659 T1975 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15663 T1976 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31450cca main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15664 T1975 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15664 T1975 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16078 T1917 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=4&state=recovering&nodeName=127.0.0.1:35237_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2000
[junit4:junit4] 2> 16943 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16944 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:46674_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46674"}
[junit4:junit4] 2> 16944 T1903 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 16944 T1903 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16946 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16946 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16947 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16947 T1972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16947 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16947 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17665 T1975 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 17666 T1975 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46674 collection:collection1 shard:shard1
[junit4:junit4] 2> 17669 T1975 oasc.ZkController.register We are http://127.0.0.1:46674/collection1/ and leader is http://127.0.0.1:35051/collection1/
[junit4:junit4] 2> 17669 T1975 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46674
[junit4:junit4] 2> 17669 T1975 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 17669 T1975 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C131 name=collection1 org.apache.solr.core.SolrCore@6b315901 url=http://127.0.0.1:46674/collection1 node=127.0.0.1:46674_ C131_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:46674_, base_url=http://127.0.0.1:46674}
[junit4:junit4] 2> 17669 T1977 C131 P46674 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 17670 T1975 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17670 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 17670 T1977 C131 P46674 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 17670 T1977 C131 P46674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 17671 T1880 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 17671 T1977 C131 P46674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 17671 T1880 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 17671 T1880 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 17674 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17675 T1880 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 17675 T1880 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 17676 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C129_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:35237_, base_url=http://127.0.0.1:35237}
[junit4:junit4] 2> 18079 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35051/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 18079 T1958 C129 P35237 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35237 START replicas=[http://127.0.0.1:35051/collection1/] nUpdates=100
[junit4:junit4] 2> 18080 T1958 C129 P35237 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 18080 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 18080 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 18080 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 18080 T1958 C129 P35237 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 18081 T1958 C129 P35237 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35051/collection1/. core=collection1
[junit4:junit4] 2> 18081 T1958 C129 P35237 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18082 T1916 C130 P35051 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 18084 T1918 C130 P35051 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 18094 T1918 C130 P35051 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> 18094 T1918 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4] 2> 18095 T1918 C130 P35051 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 18095 T1918 C130 P35051 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 18096 T1958 C129 P35237 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 18096 T1958 C129 P35237 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 18098 T1916 C130 P35051 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 18098 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 18098 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 18098 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 18100 T1918 C130 P35051 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=0
[junit4:junit4] 2> 18101 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 18101 T1958 C129 P35237 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index.20130316121743402 forceNew: false
[junit4:junit4] 2> 18101 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index.20130316121743402 lockFactory=org.apache.lucene.store.NativeFSLockFactory@51d5b69a; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 18104 T1915 C130 P35051 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 18105 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 18107 T1958 C129 P35237 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 18107 T1958 C129 P35237 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 18109 T1958 C129 P35237 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> 18109 T1958 C129 P35237 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4] 2> 18109 T1958 C129 P35237 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 18110 T1958 C129 P35237 oass.SolrIndexSearcher.<init> Opening Searcher@5147e0a7 main
[junit4:junit4] 2> 18111 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5147e0a7 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 18111 T1958 C129 P35237 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index.20130316121743402 lockFactory=org.apache.lucene.store.NativeFSLockFactory@51d5b69a; maxCacheMB=48.0 maxMergeSizeMB=4.0)
[junit4:junit4] 2> 18112 T1958 C129 P35237 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 18112 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 18112 T1958 C129 P35237 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 18112 T1958 C129 P35237 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18113 T1958 C129 P35237 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 18451 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18452 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:46674_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46674"}
[junit4:junit4] 2> 18455 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:35237_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35237"}
[junit4:junit4] 2> 18457 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18457 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18457 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18458 T1972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18458 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18458 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18675 T1917 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {coreNodeName=5&state=recovering&nodeName=127.0.0.1:46674_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 18677 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19678 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C131_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:46674_, base_url=http://127.0.0.1:46674}
[junit4:junit4] 2> 20675 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35051/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 20676 T1977 C131 P46674 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:46674 START replicas=[http://127.0.0.1:35051/collection1/] nUpdates=100
[junit4:junit4] 2> 20676 T1977 C131 P46674 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 20676 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 20677 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 20677 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 20677 T1977 C131 P46674 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 20677 T1977 C131 P46674 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35051/collection1/. core=collection1
[junit4:junit4] 2> 20677 T1977 C131 P46674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 20678 T1918 C130 P35051 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 20680 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20682 T1917 C130 P35051 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20694 T1917 C130 P35051 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> 20694 T1917 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4] 2> 20695 T1917 C130 P35051 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20695 T1917 C130 P35051 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 20696 T1977 C131 P46674 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 20696 T1977 C131 P46674 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 20698 T1918 C130 P35051 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 20699 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 20699 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 20699 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 20701 T1916 C130 P35051 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1
[junit4:junit4] 2> 20701 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 20702 T1977 C131 P46674 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index.20130316121746002 forceNew: false
[junit4:junit4] 2> 20702 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index.20130316121746002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fc1f20; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 20705 T1917 C130 P35051 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 20706 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 20708 T1977 C131 P46674 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 20708 T1977 C131 P46674 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 20709 T1977 C131 P46674 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19119e91; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19119e91; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> 20710 T1977 C131 P46674 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4] 2> 20710 T1977 C131 P46674 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 20710 T1977 C131 P46674 oass.SolrIndexSearcher.<init> Opening Searcher@577ececf main
[junit4:junit4] 2> 20711 T1976 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@577ececf main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 20711 T1977 C131 P46674 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index.20130316121746002 lockFactory=org.apache.lucene.store.NativeFSLockFactory@41fc1f20; maxCacheMB=48.0 maxMergeSizeMB=4.0)
[junit4:junit4] 2> 20712 T1977 C131 P46674 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 20712 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 20712 T1977 C131 P46674 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 20712 T1977 C131 P46674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 20713 T1977 C131 P46674 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 21464 T1903 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21465 T1903 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:46674_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46674"}
[junit4:junit4] 2> 21487 T1955 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21487 T1902 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21487 T1972 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21487 T1922 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21487 T1938 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21487 T1909 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21685 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21685 T1880 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C132 name=collection1 org.apache.solr.core.SolrCore@60c821c4 url=http://127.0.0.1:37061/collection1 node=127.0.0.1:37061_ C132_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:37061_, base_url=http://127.0.0.1:37061, leader=true}
[junit4:junit4] 2> 21690 T1897 C132 P37061 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21692 T1897 C132 P37061 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 21693 T1897 C132 P37061 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 21704 T1897 C132 P37061 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 21705 T1897 C132 P37061 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 21706 T1897 C132 P37061 oass.SolrIndexSearcher.<init> Opening Searcher@3f7480d2 main
[junit4:junit4] 2> 21706 T1897 C132 P37061 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21707 T1906 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f7480d2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 21707 T1897 C132 P37061 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> 21710 T1915 C130 P35051 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21724 T1915 C130 P35051 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4] 2> 21725 T1915 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4] 2> 21725 T1915 C130 P35051 oass.SolrIndexSearcher.<init> Opening Searcher@1cd76497 main
[junit4:junit4] 2> 21726 T1915 C130 P35051 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21726 T1924 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1cd76497 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 21727 T1915 C130 P35051 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57820/collection1/, StdNode: http://127.0.0.1:35237/collection1/, StdNode: http://127.0.0.1:46674/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> C128_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57820_, base_url=http://127.0.0.1:57820}
[junit4:junit4] 2> 21733 T1932 C128 P57820 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C131_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:46674_, base_url=http://127.0.0.1:46674}
[junit4:junit4] 2> 21734 T1967 C131 P46674 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C129_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35237_, base_url=http://127.0.0.1:35237}
[junit4:junit4] 2> 21734 T1949 C129 P35237 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 21754 T1967 C131 P46674 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19119e91; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19119e91; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4] 2> 21754 T1932 C128 P57820 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> 21755 T1967 C131 P46674 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4] 2> 21755 T1932 C128 P57820 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4] 2> 21755 T1967 C131 P46674 oass.SolrIndexSearcher.<init> Opening Searcher@4f83d4e1 main
[junit4:junit4] 2> 21755 T1932 C128 P57820 oass.SolrIndexSearcher.<init> Opening Searcher@47e1f172 main
[junit4:junit4] 2> 21756 T1967 C131 P46674 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21756 T1932 C128 P57820 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21756 T1940 oasc.JmxMonitoredMap.put WARNING Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:453)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1484)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:963)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:483)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:239)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1763)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:129)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1640)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2>
[junit4:junit4] 2> 21756 T1976 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f83d4e1 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 21757 T1940 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47e1f172 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 21757 T1967 C131 P46674 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 21757 T1932 C128 P57820 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 21764 T1949 C129 P35237 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> 21764 T1949 C129 P35237 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4] 2> 21765 T1949 C129 P35237 oass.SolrIndexSearcher.<init> Opening Searcher@5b28ad59 main
[junit4:junit4] 2> 21765 T1949 C129 P35237 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 21766 T1957 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b28ad59 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 21766 T1949 C129 P35237 /update {waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 32
[junit4:junit4] 2> 21767 T1915 C130 P35051 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 57
[junit4:junit4] 2> 21767 T1880 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21769 T1918 C130 P35051 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21771 T1934 C128 P57820 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21773 T1950 C129 P35237 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 21775 T1965 C131 P46674 REQ /select {tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 23779 T1897 C132 P37061 oass.SolrIndexSearcher.<init> Opening Searcher@759c8d99 realtime
[junit4:junit4] 2> 23780 T1897 C132 P37061 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1429677873906581504)} 0 2
[junit4:junit4] 2> 23787 T1949 C129 P35237 /update {update.distrib=FROMLEADER&_version_=-1429677873911824384&update.from=http://127.0.0.1:35051/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1429677873911824384)} 0 1
[junit4:junit4] 2> 23787 T1968 C131 P46674 /update {update.distrib=FROMLEADER&_version_=-1429677873911824384&update.from=http://127.0.0.1:35051/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1429677873911824384)} 0 1
[junit4:junit4] 2> 23788 T1933 C128 P57820 /update {update.distrib=FROMLEADER&_version_=-1429677873911824384&update.from=http://127.0.0.1:35051/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1429677873911824384)} 0 2
[junit4:junit4] 2> 23789 T1918 C130 P35051 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1429677873911824384)} 0 6
[junit4:junit4] 2> 23794 T1898 C132 P37061 /update {wt=javabin&version=2} {add=[0 (1429677873922310144)]} 0 2
[junit4:junit4] 2> 23803 T1931 C128 P57820 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1429677873926504448)]} 0 2
[junit4:junit4] 2> 23803 T1966 C131 P46674 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1429677873926504448)]} 0 2
[junit4:junit4] 2> 23803 T1950 C129 P35237 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1429677873926504448)]} 0 2
[junit4:junit4] 2> 23804 T1916 C130 P35051 /update {wt=javabin&version=2} {add=[0 (1429677873926504448)]} 0 7
[junit4:junit4] 2> 23808 T1895 C132 P37061 /update {wt=javabin&version=2} {add=[1 (1429677873936990208)]} 0 1
[junit4:junit4] 2> 23815 T1932 C128 P57820 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429677873941184512)]} 0 1
[junit4:junit4] 2> 23815 T1967 C131 P46674 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429677873941184512)]} 0 1
[junit4:junit4] 2> 23815 T1951 C129 P35237 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1429677873941184512)]} 0 1
[junit4:junit4] 2> 23816 T1917 C130 P35051 /update {wt=javabin&version=2} {add=[1 (1429677873941184512)]} 0 5
[junit4:junit4] 2> 23820 T1896 C132 P37061 /update {wt=javabin&version=2} {add=[2 (1429677873949573120)]} 0 1
[junit4:junit4] 2> 23828 T1948 C129 P35237 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1429677873954816000)]} 0 1
[junit4:junit4] 2> 23828 T1965 C131 P46674 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1429677873954816000)]} 0 1
[junit4:junit4] 2> 23829 T1915 C130 P35051 /update {test.distrib.skip.servers=http://127.0.0.1:57820/collection1/&wt=javabin&version=2} {add=[2 (1429677873954816000)]} 0 6
[junit4:junit4] 2> 23833 T1897 C132 P37061 /update {wt=javabin&version=2} {add=[3 (1429677873963204608)]} 0 1
[junit4:junit4] 2> 23840 T1968 C131 P46674 /update {distrib.from=http://127.0.0.1:35051/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1429677873967398912)]} 0 1
[junit4:junit4] 2> 23841 T1918 C130 P35051 /update {test.distrib.skip.servers=http://127.0.0.1:57820/collection1/&test.distrib.skip.servers=http://127.0.0.1:35237/collection1/&wt=javabin&version=2} {add=[3 (1429677873967398912)]} 0 5
[junit4:junit4] 2> 23844 T1898 C132 P37061 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 23884 T1898 C132 P37061 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@70f8cac7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, _0_Asserting_0.doc, _0.fdt, segments_3]
[junit4:junit4] 2> 23885 T1898 C132 P37061 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, _0_Asserting_0.doc, _0.fdt, segments_3]
[junit4:junit4] 2> 23892 T1898 C132 P37061 oass.SolrIndexSearcher.<init> Opening Searcher@65cf9869 main
[junit4:junit4] 2> 23893 T1898 C132 P37061 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 23894 T1906 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65cf9869 main{StandardDirectoryReader(segments_3:3:nrt _0(4.3):C4)}
[junit4:junit4] 2> 23895 T1898 C132 P37061 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 23898 T1916 C130 P35051 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 23944 T1916 C130 P35051 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a60f270; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6,filenames=[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, segments_6, _0_Asserting_0.doc, _0.fdt]
[junit4:junit4] 2> 23945 T1916 C130 P35051 oasc.SolrDeletionPolicy.updateCommits newest commit = 6[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, segments_6, _0_Asserting_0.doc, _0.fdt]
[junit4:junit4] 2> 23948 T1916 C130 P35051 oass.SolrIndexSearcher.<init> Opening Searcher@10a9e6ef main
[junit4:junit4] 2> 23948 T1916 C130 P35051 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 23949 T1924 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10a9e6ef main{StandardDirectoryReader(segments_6:3:nrt _0(4.3):C4)}
[junit4:junit4] 2> 23949 T1916 C130 P35051 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:57820/collection1/, StdNode: http://127.0.0.1:35237/collection1/, StdNode: http://127.0.0.1:46674/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 23951 T1949 C129 P35237 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 23951 T1966 C131 P46674 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 23951 T1934 C128 P57820 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 23998 T1949 C129 P35237 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@260c2911; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, segments_5, _0_Asserting_0.doc, _0.fdt]
[junit4:junit4] 2> 23998 T1934 C128 P57820 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1363447045305/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f7823b0; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, _0_Asserting_0.doc, _0.fdt, segments_4]
[junit4:junit4] 2> 23998 T1949 C129 P35237 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.ram, _0.nvm, _0_NestedPulsing_0.tip, _0_NestedPulsing_0.tim, _0_NestedPulsing_0.pos, _0_Pulsing41_0.doc, _0_Pulsing41_0.pos, _0_Pulsing41_0.tim, _0_Pulsing41_0.tip, _0.nvd, _0.fdx, _0.si, _0_Asserting_0.pos, segments_5, _0_Asserting_0.doc, _0.fdt]
[junit4:junit4] 2> 23998 T1934 C128 P57820 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_0_Asserting_0.tim, _0_NestedPulsing_0.doc, _0.fnm, _0_Asserting_0.tip, _0_Memory_0.
[...truncated too long message...]
junit4:junit4] 2> 48497 T1880 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=E90AE1C3E6AD6A5D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ja -Dtests.timezone=America/Recife -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 48.5s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([E90AE1C3E6AD6A5D:68EC6FDB91F20A61]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:92)
[junit4:junit4] > at org.junit.Assert.assertTrue(Assert.java:43)
[junit4:junit4] > at org.junit.Assert.assertTrue(Assert.java:54)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:196)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:662)
[junit4:junit4] 2> 48511 T1880 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 48513 T1879 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 48699 T1883 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=NestedPulsing), _version_=PostingsFormat(name=Asserting), rnd_b=Pulsing41(freqCutoff=10 minBlockSize=95 maxBlockSize=226), intDefault=PostingsFormat(name=Asserting), id=Pulsing41(freqCutoff=10 minBlockSize=95 maxBlockSize=226), timestamp=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=Pulsing41(freqCutoff=10 minBlockSize=95 maxBlockSize=226), range_facet_si=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Memory doPackFST= true), a_si=PostingsFormat(name=Asserting)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ja, timezone=America/Recife
[junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic amd64/Sun Microsystems Inc. 1.6.0_41 (64-bit)/cpus=8,threads=6,free=127267512,total=257425408
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SearchHandlerTest, PluginInfoTest, QueryElevationComponentTest, ClusterStateUpdateTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, DirectUpdateHandlerOptimizeTest, TestFuzzyAnalyzedSuggestions, ScriptEngineTest, TestCoreContainer, TestPropInjectDefaults, CircularListTest, SystemInfoHandlerTest, TestJoin, DocValuesTest, TestDocSet, UnloadDistributedZkTest, TestPHPSerializedResponseWriter, FileBasedSpellCheckerTest, BasicDistributedZkTest, TestSolrDeletionPolicy2, DistributedTermsComponentTest, TestDocumentBuilder, SuggesterFSTTest, RAMDirectoryFactoryTest, UUIDFieldTest, TestStressRecovery, DocumentBuilderTest, SolrInfoMBeanTest, IndexSchemaTest, BadComponentTest, AnalysisAfterCoreReloadTest, TestFieldCollectionResource, TestPhraseSuggestions, PolyFieldTest, LegacyHTMLStripCharFilterTest, LoggingHandlerTest, DateFieldTest, SuggesterTest, TestSearchPerf, SliceStateUpdateTest, SpellingQueryConverterTest, ShardRoutingTest, DistributedSpellCheckComponentTest, SynonymTokenizerTest, TestPseudoReturnFields, TestConfig, SimpleFacetsTest, AutoCommitTest, TestSweetSpotSimilarityFactory, RecoveryZkTest, ResponseLogComponentTest, TestFaceting, TestFunctionQuery, ExternalFileFieldSortTest, TestReversedWildcardFilterFactory, QueryEqualityTest, TestFastLRUCache, ZkControllerTest, PrimUtilsTest, CachingDirectoryFactoryTest, MoreLikeThisHandlerTest, TestRemoteStreaming, JsonLoaderTest, TestCopyFieldCollectionResource, OverseerCollectionProcessorTest, TestGroupingSearch, AliasIntegrationTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 48.78s, 1 test, 1 failure <<< FAILURES!
[...truncated 427 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1213: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:877: There were test failures: 273 suites, 1147 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 57 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 64bit/jdk1.6.0_41 -XX:+UseParallelGC
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0-ea-b79) - Build #
4702 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4702/
Java: 64bit/jdk1.8.0-ea-b79 -XX:+UseParallelGC
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=200, name=coreLoadExecutor-88-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=200, name=coreLoadExecutor-88-thread-1, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
at java.lang.Thread.getStackTrace(Thread.java:1569)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:878)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:870)
at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:34)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:244)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:458)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:744)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:619)
at org.apache.solr.core.CoreContainer.createFromZk(CoreContainer.java:872)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:955)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:579)
at org.apache.solr.core.CoreContainer$2.call(CoreContainer.java:574)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@21b2fe58 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f110421)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
... 1 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.RecoveryZkTest
Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=202, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=202, name=RecoveryThread, state=RUNNABLE, group=TGRP-RecoveryZkTest], registration stack trace below.
at java.lang.Thread.getStackTrace(Thread.java:1569)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:150)
at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:520)
at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:986)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:878)
at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:870)
at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:34)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:272)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:224)
at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:383)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:281)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223)
Caused by: java.lang.AssertionError: Directory not closed: MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:602)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:599)
at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:167)
at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:615)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:35254/lrooy/onenodecollectioncore returned non ok status:404, message:Can not find: /lrooy/onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:35254/lrooy/onenodecollectioncore returned non ok status:404, message:Can not find: /lrooy/onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([DDA8E93F6FE5B250:5C4E672718BAD26C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard
Stack Trace:
java.lang.AssertionError: Still found shard
at __randomizedtesting.SeedInfo.seed([DDA8E93F6FE5B250:5C4E672718BAD26C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 8933 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.RecoveryZkTest
[junit4:junit4] 2> 0 T141 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 4 T141 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-RecoveryZkTest-1363452959107
[junit4:junit4] 2> 5 T141 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 6 T142 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 7 T142 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 7 T142 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 8 T142 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 8 T142 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 9 T142 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-1363452959107/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 106 T141 oasc.ZkTestServer.run start zk server on port:48954
[junit4:junit4] 2> 107 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6bc7026d
[junit4:junit4] 2> 108 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108 T147 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 109 T147 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 109 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57277
[junit4:junit4] 2> 110 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57277
[junit4:junit4] 2> 110 T145 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 208 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880000 with negotiated timeout 10000 for client /127.0.0.1:57277
[junit4:junit4] 2> 208 T147 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880000, negotiated timeout = 10000
[junit4:junit4] 2> 209 T148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bc7026d name:ZooKeeperConnection Watcher:127.0.0.1:48954 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 210 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 210 T141 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 214 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880000
[junit4:junit4] 2> 215 T141 oaz.ZooKeeper.close Session: 0x13d742119880000 closed
[junit4:junit4] 2> 215 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57277 which had sessionid 0x13d742119880000
[junit4:junit4] 2> 216 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6500f165
[junit4:junit4] 2> 215 T148 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 216 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 217 T149 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 217 T149 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 217 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57278
[junit4:junit4] 2> 218 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57278
[junit4:junit4] 2> 219 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880001 with negotiated timeout 10000 for client /127.0.0.1:57278
[junit4:junit4] 2> 219 T149 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880001, negotiated timeout = 10000
[junit4:junit4] 2> 219 T150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6500f165 name:ZooKeeperConnection Watcher:127.0.0.1:48954/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 220 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 220 T141 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 224 T141 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 227 T141 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 230 T141 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 233 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 234 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 239 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 242 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 247 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 247 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 251 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 252 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 255 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 256 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 259 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 259 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 262 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 263 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 267 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 267 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 270 T141 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 271 T141 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 274 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880001
[junit4:junit4] 2> 274 T150 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 274 T141 oaz.ZooKeeper.close Session: 0x13d742119880001 closed
[junit4:junit4] 2> 274 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57278 which had sessionid 0x13d742119880001
[junit4:junit4] 2> 336 T141 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 343 T141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45612
[junit4:junit4] 2> 344 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 344 T141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 345 T141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377
[junit4:junit4] 2> 345 T141 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/solr.xml
[junit4:junit4] 2> 345 T141 oasc.CoreContainer.<init> New CoreContainer 1038946732
[junit4:junit4] 2> 346 T141 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/'
[junit4:junit4] 2> 346 T141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/'
[junit4:junit4] 2> 367 T141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 367 T141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 368 T141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 368 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 368 T141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 369 T141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 369 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 369 T141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 370 T141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 370 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 379 T141 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 391 T141 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:48954/solr
[junit4:junit4] 2> 391 T141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 392 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b9def64
[junit4:junit4] 2> 392 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 393 T160 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 393 T160 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 393 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57279
[junit4:junit4] 2> 394 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57279
[junit4:junit4] 2> 395 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880002 with negotiated timeout 20000 for client /127.0.0.1:57279
[junit4:junit4] 2> 395 T160 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880002, negotiated timeout = 20000
[junit4:junit4] 2> 395 T161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b9def64 name:ZooKeeperConnection Watcher:127.0.0.1:48954 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 395 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 397 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880002
[junit4:junit4] 2> 398 T161 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 398 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57279 which had sessionid 0x13d742119880002
[junit4:junit4] 2> 398 T141 oaz.ZooKeeper.close Session: 0x13d742119880002 closed
[junit4:junit4] 2> 399 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 402 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2f54e214
[junit4:junit4] 2> 403 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 403 T162 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 404 T162 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 404 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57280
[junit4:junit4] 2> 404 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57280
[junit4:junit4] 2> 405 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880003 with negotiated timeout 20000 for client /127.0.0.1:57280
[junit4:junit4] 2> 405 T162 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880003, negotiated timeout = 20000
[junit4:junit4] 2> 405 T163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f54e214 name:ZooKeeperConnection Watcher:127.0.0.1:48954/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 406 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 407 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 409 T141 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 411 T141 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45612_
[junit4:junit4] 2> 412 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45612_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45612_
[junit4:junit4] 2> 412 T141 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45612_
[junit4:junit4] 2> 415 T141 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 421 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 422 T141 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 424 T141 oasc.Overseer.start Overseer (id=89355253128364035-127.0.0.1:45612_-n_0000000000) starting
[junit4:junit4] 2> 425 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 426 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 426 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 427 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 428 T165 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 429 T141 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 432 T141 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 434 T141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 437 T164 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 443 T166 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/collection1
[junit4:junit4] 2> 443 T166 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 444 T166 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 445 T166 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 446 T166 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/collection1/'
[junit4:junit4] 2> 447 T166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/collection1/lib/README' to classloader
[junit4:junit4] 2> 447 T166 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 482 T166 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 523 T166 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 525 T166 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 532 T166 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 901 T166 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 909 T166 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 911 T166 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 934 T166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 938 T166 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 941 T166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 943 T166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 943 T166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 944 T166 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 945 T166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 945 T166 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 946 T166 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 946 T166 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-controljetty-1363452959377/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data/
[junit4:junit4] 2> 946 T166 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4de60052
[junit4:junit4] 2> 947 T166 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 948 T166 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data forceNew: false
[junit4:junit4] 2> 948 T166 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data/index/
[junit4:junit4] 2> 948 T166 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 948 T166 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data/index forceNew: false
[junit4:junit4] 2> 950 T166 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3544ebd9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5a2065d1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 950 T166 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 952 T166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 953 T166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 953 T166 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 954 T166 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 954 T166 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 954 T166 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 955 T166 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 955 T166 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 955 T166 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 958 T166 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 960 T166 oass.SolrIndexSearcher.<init> Opening Searcher@1169e8c7 main
[junit4:junit4] 2> 961 T166 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/control/data/tlog
[junit4:junit4] 2> 961 T166 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 962 T166 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 965 T167 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1169e8c7 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 967 T166 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 967 T166 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 968 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x55 zxid:0x2a txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 1943 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1944 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45612",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45612_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 1945 T164 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1954 T164 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1955 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x5d zxid:0x2d txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 1962 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1971 T166 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1972 T166 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45612 collection:control_collection shard:shard1
[junit4:junit4] 2> 1976 T166 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 1984 T166 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1991 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:delete cxid:0x77 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 1992 T166 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1993 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x78 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 1995 T166 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1996 T166 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1997 T166 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45612/collection1/
[junit4:junit4] 2> 1998 T166 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1998 T166 oasc.SyncStrategy.syncToMe http://127.0.0.1:45612/collection1/ has no replicas
[junit4:junit4] 2> 1998 T166 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45612/collection1/
[junit4:junit4] 2> 1999 T166 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2004 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880003 type:create cxid:0x82 zxid:0x3c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3467 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3479 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3511 T166 oasc.ZkController.register We are http://127.0.0.1:45612/collection1/ and leader is http://127.0.0.1:45612/collection1/
[junit4:junit4] 2> 3512 T166 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45612
[junit4:junit4] 2> 3512 T166 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3512 T166 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3513 T166 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3515 T166 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3517 T141 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3517 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3518 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3522 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3523 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@42b1a2ce
[junit4:junit4] 2> 3524 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3524 T169 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3525 T169 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 3525 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57282
[junit4:junit4] 2> 3526 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57282
[junit4:junit4] 2> 3527 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880004 with negotiated timeout 10000 for client /127.0.0.1:57282
[junit4:junit4] 2> 3527 T169 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880004, negotiated timeout = 10000
[junit4:junit4] 2> 3527 T170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42b1a2ce name:ZooKeeperConnection Watcher:127.0.0.1:48954/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3528 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3530 T141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3534 T141 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3594 T141 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3596 T141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42845
[junit4:junit4] 2> 3597 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3598 T141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3598 T141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636
[junit4:junit4] 2> 3598 T141 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/solr.xml
[junit4:junit4] 2> 3599 T141 oasc.CoreContainer.<init> New CoreContainer 339749098
[junit4:junit4] 2> 3599 T141 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/'
[junit4:junit4] 2> 3599 T141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/'
[junit4:junit4] 2> 3618 T141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3618 T141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3619 T141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3619 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3619 T141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3620 T141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3620 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3620 T141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3621 T141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3621 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3629 T141 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3647 T141 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:48954/solr
[junit4:junit4] 2> 3648 T141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3648 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@10343c23
[junit4:junit4] 2> 3649 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3649 T180 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3650 T180 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 3650 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57283
[junit4:junit4] 2> 3651 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57283
[junit4:junit4] 2> 3652 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880005 with negotiated timeout 20000 for client /127.0.0.1:57283
[junit4:junit4] 2> 3652 T180 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880005, negotiated timeout = 20000
[junit4:junit4] 2> 3652 T181 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10343c23 name:ZooKeeperConnection Watcher:127.0.0.1:48954 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3653 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3654 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880005
[junit4:junit4] 2> 3655 T181 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3655 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57283 which had sessionid 0x13d742119880005
[junit4:junit4] 2> 3655 T141 oaz.ZooKeeper.close Session: 0x13d742119880005 closed
[junit4:junit4] 2> 3657 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3661 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4065286
[junit4:junit4] 2> 3662 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3662 T182 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3663 T182 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 3663 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57284
[junit4:junit4] 2> 3663 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57284
[junit4:junit4] 2> 3664 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880006 with negotiated timeout 20000 for client /127.0.0.1:57284
[junit4:junit4] 2> 3665 T182 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880006, negotiated timeout = 20000
[junit4:junit4] 2> 3665 T183 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4065286 name:ZooKeeperConnection Watcher:127.0.0.1:48954/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3665 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3667 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:create cxid:0x1 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3668 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:create cxid:0x2 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3670 T141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4674 T141 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42845_
[junit4:junit4] 2> 4675 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:delete cxid:0xb zxid:0x4c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:42845_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:42845_
[junit4:junit4] 2> 4675 T141 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42845_
[junit4:junit4] 2> 4679 T183 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4679 T170 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4679 T163 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4679 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4685 T184 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/collection1
[junit4:junit4] 2> 4686 T184 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4686 T184 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4687 T184 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4688 T184 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/collection1/'
[junit4:junit4] 2> 4689 T184 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/collection1/lib/README' to classloader
[junit4:junit4] 2> 4689 T184 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4715 T184 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 4748 T184 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4749 T184 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4755 T184 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4986 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4988 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45612",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "node_name":"127.0.0.1:45612_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 4996 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4996 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4996 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5119 T184 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5125 T184 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5127 T184 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5150 T184 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5153 T184 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5157 T184 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5158 T184 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5159 T184 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5159 T184 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5160 T184 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5161 T184 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5161 T184 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5161 T184 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty1-1363452962636/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1/
[junit4:junit4] 2> 5161 T184 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4de60052
[junit4:junit4] 2> 5162 T184 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5163 T184 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1 forceNew: false
[junit4:junit4] 2> 5163 T184 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1/index/
[junit4:junit4] 2> 5163 T184 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5164 T184 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1/index forceNew: false
[junit4:junit4] 2> 5165 T184 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5640b971 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e5045db),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5166 T184 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5168 T184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5168 T184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5169 T184 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5169 T184 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5170 T184 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5170 T184 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5171 T184 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5171 T184 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5172 T184 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5174 T184 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5176 T184 oass.SolrIndexSearcher.<init> Opening Searcher@85fbdf4 main
[junit4:junit4] 2> 5177 T184 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty1/tlog
[junit4:junit4] 2> 5177 T184 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5178 T184 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5186 T185 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@85fbdf4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5186 T184 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5187 T184 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6504 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6505 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42845",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:42845_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 6505 T164 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 6505 T164 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6580 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6580 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6580 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7189 T184 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7190 T184 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42845 collection:collection1 shard:shard1
[junit4:junit4] 2> 7191 T184 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7205 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:delete cxid:0x48 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7206 T184 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7206 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:create cxid:0x49 zxid:0x5d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7239 T184 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7240 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7240 T184 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42845/collection1/
[junit4:junit4] 2> 7241 T184 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7241 T184 oasc.SyncStrategy.syncToMe http://127.0.0.1:42845/collection1/ has no replicas
[junit4:junit4] 2> 7242 T184 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42845/collection1/
[junit4:junit4] 2> 7242 T184 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7250 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880006 type:create cxid:0x53 zxid:0x61 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8085 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8097 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8097 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8098 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8121 T184 oasc.ZkController.register We are http://127.0.0.1:42845/collection1/ and leader is http://127.0.0.1:42845/collection1/
[junit4:junit4] 2> 8121 T184 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42845
[junit4:junit4] 2> 8122 T184 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8122 T184 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8122 T184 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8124 T184 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8126 T141 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 8127 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8127 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8203 T141 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8205 T141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58139
[junit4:junit4] 2> 8205 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8206 T141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8206 T141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237
[junit4:junit4] 2> 8206 T141 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/solr.xml
[junit4:junit4] 2> 8207 T141 oasc.CoreContainer.<init> New CoreContainer 1463596936
[junit4:junit4] 2> 8207 T141 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/'
[junit4:junit4] 2> 8208 T141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/'
[junit4:junit4] 2> 8225 T141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8225 T141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8226 T141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8226 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8226 T141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8227 T141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8227 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8227 T141 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8228 T141 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8228 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8234 T141 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8243 T141 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:48954/solr
[junit4:junit4] 2> 8244 T141 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8244 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@b11b129
[junit4:junit4] 2> 8245 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8245 T196 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8246 T196 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 8246 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57285
[junit4:junit4] 2> 8246 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57285
[junit4:junit4] 2> 8248 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880007 with negotiated timeout 20000 for client /127.0.0.1:57285
[junit4:junit4] 2> 8248 T196 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880007, negotiated timeout = 20000
[junit4:junit4] 2> 8249 T197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b11b129 name:ZooKeeperConnection Watcher:127.0.0.1:48954 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8249 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8251 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880007
[junit4:junit4] 2> 8258 T197 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8258 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57285 which had sessionid 0x13d742119880007
[junit4:junit4] 2> 8258 T141 oaz.ZooKeeper.close Session: 0x13d742119880007 closed
[junit4:junit4] 2> 8259 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8264 T141 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:48954/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4e9fafda
[junit4:junit4] 2> 8265 T141 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8265 T198 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:48954. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8266 T198 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:48954, initiating session
[junit4:junit4] 2> 8266 T143 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:57286
[junit4:junit4] 2> 8266 T143 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:57286
[junit4:junit4] 2> 8268 T145 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d742119880008 with negotiated timeout 20000 for client /127.0.0.1:57286
[junit4:junit4] 2> 8268 T198 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:48954, sessionid = 0x13d742119880008, negotiated timeout = 20000
[junit4:junit4] 2> 8269 T199 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e9fafda name:ZooKeeperConnection Watcher:127.0.0.1:48954/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8269 T141 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8271 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880008 type:create cxid:0x1 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8279 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880008 type:create cxid:0x2 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8283 T141 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9288 T141 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58139_
[junit4:junit4] 2> 9289 T146 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d742119880008 type:delete cxid:0xb zxid:0x70 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58139_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58139_
[junit4:junit4] 2> 9299 T141 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58139_
[junit4:junit4] 2> 9302 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9303 T170 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9303 T199 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9303 T183 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9304 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9304 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9305 T163 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9312 T200 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/collection1
[junit4:junit4] 2> 9312 T200 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9313 T200 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9313 T200 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9315 T200 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/collection1/'
[junit4:junit4] 2> 9315 T200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/collection1/lib/README' to classloader
[junit4:junit4] 2> 9315 T200 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9341 T200 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 9377 T200 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9378 T200 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9384 T200 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9606 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9607 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:42845",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "node_name":"127.0.0.1:42845_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 9612 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9612 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9612 T199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9612 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9721 T200 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9731 T200 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9734 T200 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9761 T200 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9766 T200 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9770 T200 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9772 T200 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9772 T200 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9773 T200 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9774 T200 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9774 T200 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9775 T200 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9775 T200 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/collection1/, dataDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/
[junit4:junit4] 2> 9775 T200 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4de60052
[junit4:junit4] 2> 9776 T200 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9777 T200 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2 forceNew: false
[junit4:junit4] 2> 9777 T200 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index/
[junit4:junit4] 2> 9777 T200 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9778 T200 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index forceNew: false
[junit4:junit4] 2> 9780 T200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27c5c202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c40d827),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9781 T200 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9783 T200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9784 T200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9785 T200 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9786 T200 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9786 T200 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9787 T200 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9787 T200 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9788 T200 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9788 T200 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9792 T200 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9795 T200 oass.SolrIndexSearcher.<init> Opening Searcher@64de09f1 main
[junit4:junit4] 2> 9796 T200 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/tlog
[junit4:junit4] 2> 9797 T200 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9797 T200 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9801 T201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64de09f1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9804 T200 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9804 T200 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11124 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11126 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58139",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:58139_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 11126 T164 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 11126 T164 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 11157 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11157 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11158 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11157 T199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11808 T200 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11809 T200 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58139 collection:collection1 shard:shard1
[junit4:junit4] 2> 11816 T200 oasc.ZkController.register We are http://127.0.0.1:58139/collection1/ and leader is http://127.0.0.1:42845/collection1/
[junit4:junit4] 2> 11817 T200 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58139
[junit4:junit4] 2> 11817 T200 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 11818 T200 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@3b84b706 url=http://127.0.0.1:58139/collection1 node=127.0.0.1:58139_ C7_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, base_url=http://127.0.0.1:58139, node_name=127.0.0.1:58139_, core=collection1}
[junit4:junit4] 2> 11824 T202 C7 P58139 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 11824 T200 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11825 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 11826 T202 C7 P58139 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 11827 T202 C7 P58139 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11827 T141 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 11828 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 11829 T202 C7 P58139 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11829 T141 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 11838 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C8 name=collection1 org.apache.solr.core.SolrCore@4ed2d38e url=http://127.0.0.1:45612/collection1 node=127.0.0.1:45612_ C8_STATE=coll:control_collection core:collection1 props:{state=active, collection=control_collection, base_url=http://127.0.0.1:45612, shard=shard1, node_name=127.0.0.1:45612_, core=collection1, leader=true}
[junit4:junit4] 2> 11868 T158 C8 P45612 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3544ebd9 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5a2065d1),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 11870 T158 C8 P45612 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 11878 T159 C8 P45612 /update {version=2&wt=javabin} {add=[1 (1429684062485741568)]} 0 22
[junit4:junit4] 2> 11881 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10001 (1429684062485741569)]} 0 25
[junit4:junit4] 2> ASYNC NEW_CORE C9 name=collection1 org.apache.solr.core.SolrCore@546fcbb4 url=http://127.0.0.1:42845/collection1 node=127.0.0.1:42845_ C9_STATE=coll:collection1 core:collection1 props:{state=active, collection=collection1, base_url=http://127.0.0.1:42845, shard=shard1, node_name=127.0.0.1:42845_, core=collection1, leader=true}
[junit4:junit4] 2> 11901 T177 C9 P42845 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5640b971 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e5045db),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 11903 T177 C9 P42845 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 11908 T178 C9 P42845 /update {version=2&wt=javabin} {add=[1 (1429684062520344577)]} 0 10
[junit4:junit4] 2> 11911 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10001 (1429684062520344576)]} 0 13
[junit4:junit4] 2> 11954 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[1 (-1429684062578016256)]} 0 1
[junit4:junit4] 2> 11956 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10001 (-1429684062579064832)]} 0 1
[junit4:junit4] 2> 11960 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[1 (-1429684062584307712)]} 0 1
[junit4:junit4] 2> 11961 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10001 (-1429684062585356288)]} 0 1
[junit4:junit4] 2> 11969 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10002 (1429684062590599168)]} 0 4
[junit4:junit4] 2> 11970 T157 C8 P45612 /update {version=2&wt=javabin} {add=[2 (1429684062590599169)]} 0 5
[junit4:junit4] 2> 11974 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10002 (1429684062596890624)]} 0 3
[junit4:junit4] 2> 11977 T176 C9 P42845 /update {version=2&wt=javabin} {add=[2 (1429684062597939200)]} 0 4
[junit4:junit4] 2> 11996 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10003 (1429684062617862144)]} 0 4
[junit4:junit4] 2> 11999 T159 C8 P45612 /update {version=2&wt=javabin} {add=[3 (1429684062622056448)]} 0 4
[junit4:junit4] 2> 12003 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10003 (1429684062626250752)]} 0 3
[junit4:junit4] 2> 12006 T177 C9 P42845 /update {version=2&wt=javabin} {add=[3 (1429684062629396480)]} 0 3
[junit4:junit4] 2> 12021 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10004 (1429684062644076544)]} 0 5
[junit4:junit4] 2> 12023 T157 C8 P45612 /update {version=2&wt=javabin} {add=[4 (1429684062647222272)]} 0 4
[junit4:junit4] 2> 12030 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10004 (1429684062653513728)]} 0 5
[junit4:junit4] 2> 12032 T177 C9 P42845 /update {version=2&wt=javabin} {add=[4 (1429684062655610880)]} 0 5
[junit4:junit4] 2> 12096 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10005 (1429684062724816896)]} 0 2
[junit4:junit4] 2> 12098 T159 C8 P45612 /update {version=2&wt=javabin} {add=[5 (1429684062726914048)]} 0 3
[junit4:junit4] 2> 12100 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10005 (1429684062730059776)]} 0 1
[junit4:junit4] 2> 12102 T178 C9 P42845 /update {version=2&wt=javabin} {add=[5 (1429684062732156928)]} 0 2
[junit4:junit4] 2> 12120 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10006 (1429684062749982720)]} 0 2
[junit4:junit4] 2> 12121 T157 C8 P45612 /update {version=2&wt=javabin} {add=[6 (1429684062751031296)]} 0 2
[junit4:junit4] 2> 12125 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10006 (1429684062755225600)]} 0 2
[junit4:junit4] 2> 12126 T176 C9 P42845 /update {version=2&wt=javabin} {add=[6 (1429684062756274176)]} 0 2
[junit4:junit4] 2> 12198 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10003 (-1429684062832820224)]} 0 1
[junit4:junit4] 2> 12198 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[3 (-1429684062833868800)]} 0 0
[junit4:junit4] 2> 12203 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10003 (-1429684062838063104)]} 0 1
[junit4:junit4] 2> 12203 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[3 (-1429684062839111680)]} 0 1
[junit4:junit4] 2> 12210 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10007 (1429684062843305984)]} 0 3
[junit4:junit4] 2> 12212 T157 C8 P45612 /update {version=2&wt=javabin} {add=[7 (1429684062844354560)]} 0 5
[junit4:junit4] 2> 12218 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10007 (1429684062851694592)]} 0 4
[junit4:junit4] 2> 12220 T178 C9 P42845 /update {version=2&wt=javabin} {add=[7 (1429684062853791744)]} 0 4
[junit4:junit4] 2> 12229 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10004 (-1429684062866374656)]} 0 1
[junit4:junit4] 2> 12232 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[4 (-1429684062869520384)]} 0 1
[junit4:junit4] 2> 12234 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10004 (-1429684062871617536)]} 0 1
[junit4:junit4] 2> 12238 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[4 (-1429684062874763264)]} 0 2
[junit4:junit4] 2> 12241 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10008 (1429684062876860416)]} 0 3
[junit4:junit4] 2> 12245 T157 C8 P45612 /update {version=2&wt=javabin} {add=[8 (1429684062880006144)]} 0 3
[junit4:junit4] 2> 12249 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10008 (1429684062884200448)]} 0 4
[junit4:junit4] 2> 12253 T177 C9 P42845 /update {version=2&wt=javabin} {add=[8 (1429684062888394752)]} 0 3
[junit4:junit4] 2> 12323 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10009 (1429684062962843648)]} 0 2
[junit4:junit4] 2> 12327 T159 C8 P45612 /update {version=2&wt=javabin} {add=[9 (1429684062967037952)]} 0 3
[junit4:junit4] 2> 12329 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10009 (1429684062969135104)]} 0 2
[junit4:junit4] 2> 12333 T177 C9 P42845 /update {version=2&wt=javabin} {add=[9 (1429684062973329408)]} 0 3
[junit4:junit4] 2> 12435 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10010 (1429684063079235584)]} 0 4
[junit4:junit4] 2> 12439 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10 (1429684063084478464)]} 0 4
[junit4:junit4] 2> 12443 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10010 (1429684063087624192)]} 0 3
[junit4:junit4] 2> 12447 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10 (1429684063091818496)]} 0 4
[junit4:junit4] 2> 12486 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10007 (-1429684063135858688)]} 0 1
[junit4:junit4] 2> 12489 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[7 (-1429684063139004416)]} 0 0
[junit4:junit4] 2> 12490 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[10007 (-1429684063140052992)]} 0 0
[junit4:junit4] 2> 12494 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[7 (-1429684063144247296)]} 0 1
[junit4:junit4] 2> 12499 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10011 (1429684063145295872)]} 0 5
[junit4:junit4] 2> 12501 T156 C8 P45612 /update {version=2&wt=javabin} {add=[11 (1429684063148441600)]} 0 4
[junit4:junit4] 2> 12506 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10011 (1429684063154733056)]} 0 3
[junit4:junit4] 2> 12508 T176 C9 P42845 /update {version=2&wt=javabin} {add=[11 (1429684063155781632)]} 0 3
[junit4:junit4] 2> 12552 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10009 (-1429684063205064704)]} 0 1
[junit4:junit4] 2> 12554 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[9 (-1429684063207161856)]} 0 1
[junit4:junit4] 2> 12556 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10009 (-1429684063209259008)]} 0 0
[junit4:junit4] 2> 12558 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[9 (-1429684063211356160)]} 0 0
[junit4:junit4] 2> 12564 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10012 (1429684063214501888)]} 0 4
[junit4:junit4] 2> 12566 T156 C8 P45612 /update {version=2&wt=javabin} {add=[12 (1429684063216599040)]} 0 4
[junit4:junit4] 2> 12571 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10012 (1429684063221841920)]} 0 3
[junit4:junit4] 2> 12574 T178 C9 P42845 /update {version=2&wt=javabin} {add=[12 (1429684063224987648)]} 0 4
[junit4:junit4] 2> 12631 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10013 (1429684063286853632)]} 0 2
[junit4:junit4] 2> 12632 T159 C8 P45612 /update {version=2&wt=javabin} {add=[13 (1429684063287902208)]} 0 1
[junit4:junit4] 2> 12636 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10013 (1429684063291047936)]} 0 2
[junit4:junit4] 2> 12637 T176 C9 P42845 /update {version=2&wt=javabin} {add=[13 (1429684063292096512)]} 0 2
[junit4:junit4] 2> 12665 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12666 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58139",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58139_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 12671 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12671 T199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12671 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12671 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12735 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10014 (1429684063393808384)]} 0 4
[junit4:junit4] 2> 12737 T156 C8 P45612 /update {version=2&wt=javabin} {add=[14 (1429684063393808385)]} 0 5
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=recovering, collection=collection1, base_url=http://127.0.0.1:58139, shard=shard1, node_name=127.0.0.1:58139_, core=collection1}
[junit4:junit4] 2> 12765 T193 C7 P58139 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27c5c202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c40d827),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 12766 T193 C7 P58139 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 12770 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10014 (1429684063407439873)]} 0 7
[junit4:junit4] 2> 12771 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[14 (1429684063407439872)]} 0 8
[junit4:junit4] 2> 12771 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10014 (1429684063407439873)]} 0 32
[junit4:junit4] 2> 12772 T177 C9 P42845 /update {version=2&wt=javabin} {add=[14 (1429684063407439872)]} 0 29
[junit4:junit4] 2> 12803 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10015 (1429684063465111552)]} 0 3
[junit4:junit4] 2> 12805 T159 C8 P45612 /update {version=2&wt=javabin} {add=[15 (1429684063465111553)]} 0 5
[junit4:junit4] 2> 12816 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10015 (1429684063472451584)]} 0 3
[junit4:junit4] 2> 12817 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10015 (1429684063472451584)]} 0 11
[junit4:junit4] 2> 12818 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[15 (1429684063474548736)]} 0 3
[junit4:junit4] 2> 12819 T178 C9 P42845 /update {version=2&wt=javabin} {add=[15 (1429684063474548736)]} 0 11
[junit4:junit4] 2> 12850 T179 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {onlyIfLeader=true&checkLive=true&state=recovering&nodeName=127.0.0.1:58139_&version=2&wt=javabin&action=PREPRECOVERY&coreNodeName=3&core=collection1} status=0 QTime=1003
[junit4:junit4] 2> 12895 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[10010 (-1429684063564726272)]} 0 1
[junit4:junit4] 2> 12896 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[10 (-1429684063565774848)]} 0 0
[junit4:junit4] 2> 12903 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10010 (-1429684063567872000)]} 0 1
[junit4:junit4] 2> 12903 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10 (-1429684063568920576)]} 0 1
[junit4:junit4] 2> 12904 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10010 (-1429684063567872000)]} 0 6
[junit4:junit4] 2> 12904 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10 (-1429684063568920576)]} 0 5
[junit4:junit4] 2> 12908 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10016 (1429684063577309184)]} 0 2
[junit4:junit4] 2> 12909 T157 C8 P45612 /update {version=2&wt=javabin} {add=[16 (1429684063577309185)]} 0 2
[junit4:junit4] 2> 12919 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10016 (1429684063582552064)]} 0 2
[junit4:junit4] 2> 12920 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10016 (1429684063582552064)]} 0 9
[junit4:junit4] 2> 12921 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[16 (1429684063583600640)]} 0 3
[junit4:junit4] 2> 12922 T176 C9 P42845 /update {version=2&wt=javabin} {add=[16 (1429684063583600640)]} 0 9
[junit4:junit4] 2> 13007 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10017 (1429684063677972480)]} 0 4
[junit4:junit4] 2> 13009 T158 C8 P45612 /update {version=2&wt=javabin} {add=[17 (1429684063680069632)]} 0 5
[junit4:junit4] 2> 13023 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10017 (1429684063687409664)]} 0 5
[junit4:junit4] 2> 13025 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10017 (1429684063687409664)]} 0 14
[junit4:junit4] 2> 13030 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[17 (1429684063689506816)]} 0 5
[junit4:junit4] 2> 13031 T179 C9 P42845 /update {version=2&wt=javabin} {add=[17 (1429684063689506816)]} 0 18
[junit4:junit4] 2> 13069 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10018 (1429684063744032768)]} 0 4
[junit4:junit4] 2> 13073 T157 C8 P45612 /update {version=2&wt=javabin} {add=[18 (1429684063748227072)]} 0 3
[junit4:junit4] 2> 13083 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10018 (1429684063751372800)]} 0 4
[junit4:junit4] 2> 13084 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10018 (1429684063751372800)]} 0 11
[junit4:junit4] 2> 13088 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[18 (1429684063755567104)]} 0 5
[junit4:junit4] 2> 13090 T176 C9 P42845 /update {version=2&wt=javabin} {add=[18 (1429684063755567104)]} 0 13
[junit4:junit4] 2> 13102 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10011 (-1429684063781781504)]} 0 1
[junit4:junit4] 2> 13107 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[11 (-1429684063787024384)]} 0 0
[junit4:junit4] 2> 13110 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10011 (-1429684063787024384)]} 0 0
[junit4:junit4] 2> 13111 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10011 (-1429684063787024384)]} 0 5
[junit4:junit4] 2> 13114 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[11 (-1429684063792267264)]} 0 0
[junit4:junit4] 2> 13115 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[11 (-1429684063792267264)]} 0 4
[junit4:junit4] 2> 13115 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10019 (1429684063794364416)]} 0 1
[junit4:junit4] 2> 13120 T157 C8 P45612 /update {version=2&wt=javabin} {add=[19 (1429684063798558720)]} 0 2
[junit4:junit4] 2> 13124 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10019 (1429684063798558720)]} 0 2
[junit4:junit4] 2> 13125 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10019 (1429684063798558720)]} 0 7
[junit4:junit4] 2> 13129 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[19 (1429684063803801600)]} 0 2
[junit4:junit4] 2> 13131 T176 C9 P42845 /update {version=2&wt=javabin} {add=[19 (1429684063803801600)]} 0 7
[junit4:junit4] 2> 13221 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10015 (-1429684063906562048)]} 0 0
[junit4:junit4] 2> 13227 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[15 (-1429684063912853504)]} 0 1
[junit4:junit4] 2> 13227 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10015 (-1429684063909707776)]} 0 0
[junit4:junit4] 2> 13228 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10015 (-1429684063909707776)]} 0 4
[junit4:junit4] 2> 13232 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[15 (-1429684063915999232)]} 0 0
[junit4:junit4] 2> 13233 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[15 (-1429684063915999232)]} 0 3
[junit4:junit4] 2> 13233 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10020 (1429684063917047808)]} 0 2
[junit4:junit4] 2> 13238 T157 C8 P45612 /update {version=2&wt=javabin} {add=[20 (1429684063922290688)]} 0 2
[junit4:junit4] 2> 13241 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10020 (1429684063922290688)]} 0 2
[junit4:junit4] 2> 13242 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10020 (1429684063922290688)]} 0 6
[junit4:junit4] 2> 13247 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[20 (1429684063927533568)]} 0 1
[junit4:junit4] 2> 13248 T178 C9 P42845 /update {version=2&wt=javabin} {add=[20 (1429684063927533568)]} 0 7
[junit4:junit4] 2> 13346 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10021 (1429684064035536896)]} 0 2
[junit4:junit4] 2> 13354 T156 C8 P45612 /update {version=2&wt=javabin} {add=[21 (1429684064043925504)]} 0 2
[junit4:junit4] 2> 13357 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10021 (1429684064040779776)]} 0 2
[junit4:junit4] 2> 13358 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10021 (1429684064040779776)]} 0 9
[junit4:junit4] 2> 13364 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[21 (1429684064050216960)]} 0 2
[junit4:junit4] 2> 13365 T176 C9 P42845 /update {version=2&wt=javabin} {add=[21 (1429684064050216960)]} 0 8
[junit4:junit4] 2> 13414 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10016 (-1429684064108937216)]} 0 1
[junit4:junit4] 2> 13420 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10016 (-1429684064112082944)]} 0 0
[junit4:junit4] 2> 13420 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[16 (-1429684064115228672)]} 0 0
[junit4:junit4] 2> 13421 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10016 (-1429684064112082944)]} 0 4
[junit4:junit4] 2> 13427 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10022 (1429684064120471552)]} 0 3
[junit4:junit4] 2> 13427 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[16 (-1429684064119422976)]} 0 1
[junit4:junit4] 2> 13428 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[16 (-1429684064119422976)]} 0 5
[junit4:junit4] 2> 13434 T156 C8 P45612 /update {version=2&wt=javabin} {add=[22 (1429684064127811584)]} 0 2
[junit4:junit4] 2> 13439 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10022 (1429684064126763008)]} 0 3
[junit4:junit4] 2> 13441 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10022 (1429684064126763008)]} 0 10
[junit4:junit4] 2> 13447 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[22 (1429684064134103040)]} 0 3
[junit4:junit4] 2> 13448 T176 C9 P42845 /update {version=2&wt=javabin} {add=[22 (1429684064134103040)]} 0 10
[junit4:junit4] 2> 13497 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10017 (-1429684064195969024)]} 0 0
[junit4:junit4] 2> 13504 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[17 (-1429684064203309056)]} 0 0
[junit4:junit4] 2> 13504 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10017 (-1429684064201211904)]} 0 0
[junit4:junit4] 2> 13513 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10017 (-1429684064201211904)]} 0 12
[junit4:junit4] 2> 13519 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[17 (-1429684064215891968)]} 0 1
[junit4:junit4] 2> 13519 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10023 (1429684064216940544)]} 0 2
[junit4:junit4] 2> 13520 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[17 (-1429684064215891968)]} 0 4
[junit4:junit4] 2> 13526 T157 C8 P45612 /update {version=2&wt=javabin} {add=[23 (1429684064224280576)]} 0 2
[junit4:junit4] 2> 13529 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10023 (1429684064223232000)]} 0 2
[junit4:junit4] 2> 13530 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10023 (1429684064223232000)]} 0 8
[junit4:junit4] 2> 13536 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[23 (1429684064230572032)]} 0 2
[junit4:junit4] 2> 13537 T176 C9 P42845 /update {version=2&wt=javabin} {add=[23 (1429684064230572032)]} 0 7
[junit4:junit4] 2> 13558 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10024 (1429684064258883584)]} 0 2
[junit4:junit4] 2> 13566 T159 C8 P45612 /update {version=2&wt=javabin} {add=[24 (1429684064266223616)]} 0 2
[junit4:junit4] 2> 13568 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10024 (1429684064264126464)]} 0 2
[junit4:junit4] 2> 13570 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10024 (1429684064264126464)]} 0 8
[junit4:junit4] 2> 13576 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[24 (1429684064272515072)]} 0 2
[junit4:junit4] 2> 13577 T179 C9 P42845 /update {version=2&wt=javabin} {add=[24 (1429684064272515072)]} 0 8
[junit4:junit4] 2> 13657 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10025 (1429684064362692608)]} 0 2
[junit4:junit4] 2> 13664 T157 C8 P45612 /update {version=2&wt=javabin} {add=[25 (1429684064368984064)]} 0 2
[junit4:junit4] 2> 13667 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10025 (1429684064367935488)]} 0 2
[junit4:junit4] 2> 13668 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10025 (1429684064367935488)]} 0 8
[junit4:junit4] 2> 13673 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[25 (1429684064374226944)]} 0 3
[junit4:junit4] 2> 13674 T176 C9 P42845 /update {version=2&wt=javabin} {add=[25 (1429684064374226944)]} 0 8
[junit4:junit4] 2> 13677 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[10021 (-1429684064384712704)]} 0 0
[junit4:junit4] 2> 13684 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10021 (-1429684064388907008)]} 0 0
[junit4:junit4] 2> 13684 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[21 (-1429684064392052736)]} 0 0
[junit4:junit4] 2> 13685 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10021 (-1429684064388907008)]} 0 4
[junit4:junit4] 2> 13691 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[21 (-1429684064396247040)]} 0 1
[junit4:junit4] 2> 13691 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10026 (1429684064398344192)]} 0 2
[junit4:junit4] 2> 13692 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[21 (-1429684064396247040)]} 0 4
[junit4:junit4] 2> 13699 T157 C8 P45612 /update {version=2&wt=javabin} {add=[26 (1429684064406732800)]} 0 2
[junit4:junit4] 2> 13702 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10026 (1429684064403587072)]} 0 2
[junit4:junit4] 2> 13703 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10026 (1429684064403587072)]} 0 8
[junit4:junit4] 2> 13709 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[26 (1429684064411975680)]} 0 2
[junit4:junit4] 2> 13711 T176 C9 P42845 /update {version=2&wt=javabin} {add=[26 (1429684064411975680)]} 0 9
[junit4:junit4] 2> 13721 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[10026 (-1429684064429801472)]} 0 1
[junit4:junit4] 2> 13727 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10026 (-1429684064433995776)]} 0 0
[junit4:junit4] 2> 13727 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[26 (-1429684064437141504)]} 0 0
[junit4:junit4] 2> 13728 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10026 (-1429684064433995776)]} 0 4
[junit4:junit4] 2> 13732 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10027 (1429684064441335808)]} 0 1
[junit4:junit4] 2> 13737 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[26 (-1429684064440287232)]} 0 0
[junit4:junit4] 2> 13738 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[26 (-1429684064440287232)]} 0 8
[junit4:junit4] 2> 13743 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10027 (1429684064447627264)]} 0 2
[junit4:junit4] 2> 13743 T157 C8 P45612 /update {version=2&wt=javabin} {add=[27 (1429684064452870144)]} 0 2
[junit4:junit4] 2> 13745 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10027 (1429684064447627264)]} 0 8
[junit4:junit4] 2> 13755 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[27 (1429684064459161600)]} 0 3
[junit4:junit4] 2> 13756 T176 C9 P42845 /update {version=2&wt=javabin} {add=[27 (1429684064459161600)]} 0 9
[junit4:junit4] 2> 13801 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10028 (1429684064512638976)]} 0 2
[junit4:junit4] 2> 13810 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10028 (1429684064517881856)]} 0 2
[junit4:junit4] 2> 13811 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10028 (1429684064517881856)]} 0 7
[junit4:junit4] 2> 13811 T156 C8 P45612 /update {version=2&wt=javabin} {add=[28 (1429684064524173312)]} 0 1
[junit4:junit4] 2> 13821 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[28 (1429684064529416192)]} 0 2
[junit4:junit4] 2> 13822 T178 C9 P42845 /update {version=2&wt=javabin} {add=[28 (1429684064529416192)]} 0 8
[junit4:junit4] 2> 13825 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10029 (1429684064537804800)]} 0 2
[junit4:junit4] 2> 13836 T158 C8 P45612 /update {version=2&wt=javabin} {add=[29 (1429684064549339136)]} 0 2
[junit4:junit4] 2> 13838 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10029 (1429684064543047680)]} 0 6
[junit4:junit4] 2> 13839 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10029 (1429684064543047680)]} 0 11
[junit4:junit4] 2> 13843 T141 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 58139
[junit4:junit4] 2> 13844 T141 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1463596936
[junit4:junit4] 2> 13847 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[29 (1429684064555630592)]} 0 3
[junit4:junit4] 2> 13848 T176 C9 P42845 /update {version=2&wt=javabin} {add=[29 (1429684064555630592)]} 0 9
[junit4:junit4] 2> 13942 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10030 (1429684064661536768)]} 0 2
[junit4:junit4] 2> 13950 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10030 (1429684064666779648)]} 0 1
[junit4:junit4] 2> 13951 T157 C8 P45612 /update {version=2&wt=javabin} {add=[30 (1429684064669925376)]} 0 2
[junit4:junit4] 2> 13952 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10030 (1429684064666779648)]} 0 7
[junit4:junit4] 2> 13957 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10031 (1429684064676216832)]} 0 2
[junit4:junit4] 2> 13960 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[30 (1429684064675168256)]} 0 1
[junit4:junit4] 2> 13960 T179 C9 P42845 /update {version=2&wt=javabin} {add=[30 (1429684064675168256)]} 0 6
[junit4:junit4] 2> 13965 T159 C8 P45612 /update {version=2&wt=javabin} {add=[31 (1429684064685654016)]} 0 2
[junit4:junit4] 2> 13965 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10031 (1429684064681459712)]} 0 1
[junit4:junit4] 2> 13966 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10031 (1429684064681459712)]} 0 6
[junit4:junit4] 2> 13979 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[31 (1429684064689848320)]} 0 2
[junit4:junit4] 2> 13980 T176 C9 P42845 /update {version=2&wt=javabin} {add=[31 (1429684064689848320)]} 0 12
[junit4:junit4] 2> 14009 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10032 (1429684064731791360)]} 0 2
[junit4:junit4] 2> 14017 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10032 (1429684064735985664)]} 0 2
[junit4:junit4] 2> 14018 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10032 (1429684064735985664)]} 0 5
[junit4:junit4] 2> 14023 T157 C8 P45612 /update {version=2&wt=javabin} {add=[32 (1429684064746471424)]} 0 2
[junit4:junit4] 2> 14032 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[32 (1429684064750665728)]} 0 2
[junit4:junit4] 2> 14033 T179 C9 P42845 /update {version=2&wt=javabin} {add=[32 (1429684064750665728)]} 0 7
[junit4:junit4] 2> 14047 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[10027 (-1429684064772685824)]} 0 1
[junit4:junit4] 2> 14053 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10027 (-1429684064775831552)]} 0 1
[junit4:junit4] 2> 14054 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10027 (-1429684064775831552)]} 0 4
[junit4:junit4] 2> 14059 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10033 (1429684064783171584)]} 0 2
[junit4:junit4] 2> 14062 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[27 (-1429684064788414464)]} 0 0
[junit4:junit4] 2> 14067 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10033 (1429684064788414464)]} 0 1
[junit4:junit4] 2> 14068 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10033 (1429684064788414464)]} 0 7
[junit4:junit4] 2> 14069 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[27 (-1429684064792608768)]} 0 1
[junit4:junit4] 2> 14070 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[27 (-1429684064792608768)]} 0 5
[junit4:junit4] 2> 14075 T157 C8 P45612 /update {version=2&wt=javabin} {add=[33 (1429684064799948800)]} 0 2
[junit4:junit4] 2> 14084 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[33 (1429684064805191680)]} 0 2
[junit4:junit4] 2> 14085 T177 C9 P42845 /update {version=2&wt=javabin} {add=[33 (1429684064805191680)]} 0 7
[junit4:junit4] 2> 14120 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10034 (1429684064848183296)]} 0 1
[junit4:junit4] 2> 14128 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10034 (1429684064853426176)]} 0 1
[junit4:junit4] 2> 14129 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10034 (1429684064853426176)]} 0 6
[junit4:junit4] 2> 14138 T159 C8 P45612 /update {version=2&wt=javabin} {add=[34 (1429684064866009088)]} 0 2
[junit4:junit4] 2> 14147 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[34 (1429684064871251968)]} 0 2
[junit4:junit4] 2> 14148 T179 C9 P42845 /update {version=2&wt=javabin} {add=[34 (1429684064871251968)]} 0 7
[junit4:junit4] 2> 14176 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14177 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58139",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58139_",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 14191 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14192 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14192 T199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14191 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 14230 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10028 (-1429684064964575232)]} 0 0
[junit4:junit4] 2> 14233 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10028 (-1429684064967720960)]} 0 0
[junit4:junit4] 2> 14237 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10035 (1429684064970866688)]} 0 1
[junit4:junit4] 2> 14241 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10035 (1429684064975060992)]} 0 1
[junit4:junit4] 2> 14249 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[28 (-1429684064984498176)]} 0 0
[junit4:junit4] 2> 14253 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[28 (-1429684064987643904)]} 0 1
[junit4:junit4] 2> 14257 T157 C8 P45612 /update {version=2&wt=javabin} {add=[35 (1429684064991838208)]} 0 2
[junit4:junit4] 2> 14261 T178 C9 P42845 /update {version=2&wt=javabin} {add=[35 (1429684064996032512)]} 0 1
[junit4:junit4] 2> 14311 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10030 (-1429684065049509888)]} 0 0
[junit4:junit4] 2> 14316 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10030 (-1429684065053704192)]} 0 1
[junit4:junit4] 2> 14320 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10036 (1429684065057898496)]} 0 1
[junit4:junit4] 2> 14324 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10036 (1429684065062092800)]} 0 2
[junit4:junit4] 2> 14331 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[30 (-1429684065070481408)]} 0 0
[junit4:junit4] 2> 14334 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[30 (-1429684065073627136)]} 0 0
[junit4:junit4] 2> 14338 T157 C8 P45612 /update {version=2&wt=javabin} {add=[36 (1429684065076772864)]} 0 1
[junit4:junit4] 2> 14342 T178 C9 P42845 /update {version=2&wt=javabin} {add=[36 (1429684065080967168)]} 0 1
[junit4:junit4] 2> 14345 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10037 (1429684065084112896)]} 0 1
[junit4:junit4] 2> 14350 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10037 (1429684065089355776)]} 0 2
[junit4:junit4] 2> 14365 T158 C8 P45612 /update {version=2&wt=javabin} {add=[37 (1429684065105084416)]} 0 2
[junit4:junit4] 2> 14369 T176 C9 P42845 /update {version=2&wt=javabin} {add=[37 (1429684065109278720)]} 0 2
[junit4:junit4] 2> 14382 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10035 (-1429684065123958784)]} 0 1
[junit4:junit4] 2> 14384 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10035 (-1429684065126055936)]} 0 0
[junit4:junit4] 2> 14388 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10038 (1429684065129201664)]} 0 1
[junit4:junit4] 2> 14393 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10038 (1429684065133395968)]} 0 2
[junit4:junit4] 2> 14401 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[35 (-1429684065142833152)]} 0 1
[junit4:junit4] 2> 14405 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[35 (-1429684065148076032)]} 0 1
[junit4:junit4] 2> 14411 T158 C8 P45612 /update {version=2&wt=javabin} {add=[38 (1429684065152270336)]} 0 2
[junit4:junit4] 2> 14416 T176 C9 P42845 /update {version=2&wt=javabin} {add=[38 (1429684065157513216)]} 0 2
[junit4:junit4] 2> 14492 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10036 (-1429684065239302144)]} 0 0
[junit4:junit4] 2> 14495 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10036 (-1429684065242447872)]} 0 0
[junit4:junit4] 2> 14499 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10039 (1429684065244545024)]} 0 2
[junit4:junit4] 2> 14502 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10039 (1429684065248739328)]} 0 1
[junit4:junit4] 2> 14515 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[36 (-1429684065263419392)]} 0 0
[junit4:junit4] 2> 14518 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[36 (-1429684065266565120)]} 0 0
[junit4:junit4] 2> 14522 T158 C8 P45612 /update {version=2&wt=javabin} {add=[39 (1429684065269710848)]} 0 2
[junit4:junit4] 2> 14526 T176 C9 P42845 /update {version=2&wt=javabin} {add=[39 (1429684065273905152)]} 0 1
[junit4:junit4] 2> 14565 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10040 (1429684065314799616)]} 0 2
[junit4:junit4] 2> 14569 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10040 (1429684065318993920)]} 0 1
[junit4:junit4] 2> 14584 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10041 (1429684065334722560)]} 0 2
[junit4:junit4] 2> 14588 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10041 (1429684065338916864)]} 0 1
[junit4:junit4] 2> 14588 T156 C8 P45612 /update {version=2&wt=javabin} {add=[40 (1429684065338916864)]} 0 1
[junit4:junit4] 2> 14592 T177 C9 P42845 /update {version=2&wt=javabin} {add=[40 (1429684065343111168)]} 0 1
[junit4:junit4] 2> 14608 T158 C8 P45612 /update {version=2&wt=javabin} {add=[41 (1429684065358839808)]} 0 2
[junit4:junit4] 2> 14612 T176 C9 P42845 /update {version=2&wt=javabin} {add=[41 (1429684065364082688)]} 0 2
[junit4:junit4] 2> 14651 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10042 (1429684065404977152)]} 0 1
[junit4:junit4] 2> 14655 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10042 (1429684065409171456)]} 0 2
[junit4:junit4] 2> 14675 T156 C8 P45612 /update {version=2&wt=javabin} {add=[42 (1429684065429094400)]} 0 2
[junit4:junit4] 2> 14679 T177 C9 P42845 /update {version=2&wt=javabin} {add=[42 (1429684065434337280)]} 0 2
[junit4:junit4] 2> 14746 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10043 (1429684065504591872)]} 0 1
[junit4:junit4] 2> 14750 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10043 (1429684065508786176)]} 0 1
[junit4:junit4] 2> 14767 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10043 (-1429684065526611968)]} 0 1
[junit4:junit4] 2> 14769 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10043 (-1429684065529757696)]} 0 0
[junit4:junit4] 2> 14770 T156 C8 P45612 /update {version=2&wt=javabin} {add=[43 (1429684065529757696)]} 0 1
[junit4:junit4] 2> 14773 T179 C9 P42845 /update {version=2&wt=javabin} {add=[43 (1429684065532903424)]} 0 1
[junit4:junit4] 2> 14774 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10044 (1429684065532903424)]} 0 2
[junit4:junit4] 2> 14778 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10044 (1429684065538146304)]} 0 1
[junit4:junit4] 2> 14790 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[43 (-1429684065551777792)]} 0 0
[junit4:junit4] 2> 14793 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[43 (-1429684065554923520)]} 0 0
[junit4:junit4] 2> 14797 T159 C8 P45612 /update {version=2&wt=javabin} {add=[44 (1429684065558069248)]} 0 1
[junit4:junit4] 2> 14801 T178 C9 P42845 /update {version=2&wt=javabin} {add=[44 (1429684065562263552)]} 0 1
[junit4:junit4] 2> 14843 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10045 (1429684065606303744)]} 0 1
[junit4:junit4] 2> 14847 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10045 (1429684065610498048)]} 0 1
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, base_url=http://127.0.0.1:58139, shard=shard1, node_name=127.0.0.1:58139_, core=collection1}
[junit4:junit4] 2> 14851 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:42845/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 14851 T202 C7 P58139 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58139 START replicas=[http://127.0.0.1:42845/collection1/] nUpdates=100
[junit4:junit4] 2> 14852 T202 C7 P58139 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 14853 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 14853 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 14853 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 14854 T202 C7 P58139 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/tlog/tlog.0000000000000000000 refcount=1}}
[junit4:junit4] 2> 14854 T202 C7 P58139 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:42845/collection1/. core=collection1
[junit4:junit4] 2> 14854 T202 C7 P58139 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 14854 T176 C9 P42845 REQ /get {getVersions=100&version=2&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 14859 T178 C9 P42845 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 14866 T158 C8 P45612 /update {version=2&wt=javabin} {add=[45 (1429684065629372416)]} 0 2
[junit4:junit4] 2> 14871 T179 C9 P42845 /update {version=2&wt=javabin} {add=[45 (1429684065634615296)]} 0 2
[junit4:junit4] 2> 14915 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10044 (-1429684065682849792)]} 0 0
[junit4:junit4] 2> 14919 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10044 (-1429684065687044096)]} 0 1
[junit4:junit4] 2> 14919 T178 C9 P42845 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5640b971 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e5045db),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5640b971 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3e5045db),segFN=segments_2,generation=2,filenames=[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_2, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0_MockFixedIntBlock_0.frq, _0.si, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0.fnm, _0_Lucene41_0.doc, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 14920 T178 C9 P42845 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_2, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0_MockFixedIntBlock_0.frq, _0.si, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0.fnm, _0_Lucene41_0.doc, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 14924 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10046 (1429684065690189824)]} 0 2
[junit4:junit4] 2> 14928 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10046 (1429684065695432704)]} 0 2
[junit4:junit4] 2> 14939 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[44 (-1429684065708015616)]} 0 0
[junit4:junit4] 2> 14942 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[44 (-1429684065711161344)]} 0 0
[junit4:junit4] 2> 14947 T158 C8 P45612 /update {version=2&wt=javabin} {add=[46 (1429684065714307072)]} 0 2
[junit4:junit4] 2> 14951 T177 C9 P42845 /update {version=2&wt=javabin} {add=[46 (1429684065719549952)]} 0 2
[junit4:junit4] 2> 14955 T178 C9 P42845 oass.SolrIndexSearcher.<init> Opening Searcher@103021f5 realtime
[junit4:junit4] 2> 14956 T178 C9 P42845 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 14956 T178 C9 P42845 /update {waitSearcher=true&commit_end_point=true&version=2&wt=javabin&softCommit=false&openSearcher=false&commit=true} {commit=} 0 97
[junit4:junit4] 2> 14974 T202 C7 P58139 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14974 T202 C7 P58139 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 14979 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10045 (-1429684065748910080)]} 0 1
[junit4:junit4] 2> 14979 T202 C7 P58139 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 14982 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10045 (-1429684065753104384)]} 0 1
[junit4:junit4] 2> 14984 T179 C9 P42845 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14985 T179 C9 P42845 REQ /replication {command=indexversion&version=2&qt=/replication&wt=javabin} status=0 QTime=4
[junit4:junit4] 2> 14986 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 14986 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 14987 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 14986 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10047 (1429684065756250112)]} 0 1
[junit4:junit4] 2> 14989 T178 C9 P42845 REQ /replication {command=filelist&generation=2&version=2&qt=/replication&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 14991 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 27
[junit4:junit4] 2> 14991 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10047 (1429684065760444416)]} 0 2
[junit4:junit4] 2> 14991 T202 C7 P58139 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index.20130316235614093 forceNew: false
[junit4:junit4] 2> 14992 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d) fullCopy=true
[junit4:junit4] 2> 14998 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_Direct_0.doc&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=2
[junit4:junit4] 2> 15001 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[45 (-1429684065773027328)]} 0 0
[junit4:junit4] 2> 15004 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[45 (-1429684065776173056)]} 0 0
[junit4:junit4] 2> 15005 T178 C9 P42845 REQ /replication {command=filecontent&file=_0_Lucene41_0.pos&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15008 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.pos&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15009 T158 C8 P45612 /update {version=2&wt=javabin} {add=[47 (1429684065779318784)]} 0 2
[junit4:junit4] 2> 15011 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_Asserting_0.tim&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15013 T179 C9 P42845 /update {version=2&wt=javabin} {add=[47 (1429684065784561664)]} 0 2
[junit4:junit4] 2> 15014 T178 C9 P42845 REQ /replication {command=filecontent&file=segments_2&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15018 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.tii&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15022 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.tib&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15025 T179 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.skp&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15029 T178 C9 P42845 REQ /replication {command=filecontent&file=_0_Asserting_0.tip&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15032 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_1.del&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15035 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_Direct_0.tim&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15038 T179 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.doc&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15042 T178 C9 P42845 REQ /replication {command=filecontent&file=_0_Direct_0.pos&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15045 T176 C9 P42845 REQ /replication {command=filecontent&file=_0.si&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15048 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.frq&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15052 T179 C9 P42845 REQ /replication {command=filecontent&file=_0_Lucene41_0.tim&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15055 T178 C9 P42845 REQ /replication {command=filecontent&file=_0_MockFixedIntBlock_0.pyl&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15056 T202 C7 P58139 oash.SnapPuller$DirectoryFileFetcher.fetchPackets WARNING No content recieved for file: {size=0, name=_0_MockFixedIntBlock_0.pyl}
[junit4:junit4] 2> 15059 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_Direct_0.tip&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15063 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_Lucene41_0.doc&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15067 T179 C9 P42845 REQ /replication {command=filecontent&file=_0.fnm&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15071 T178 C9 P42845 REQ /replication {command=filecontent&file=_0.fdt&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15074 T176 C9 P42845 REQ /replication {command=filecontent&file=_0.nvd&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15075 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10046 (-1429684065850621952)]} 0 0
[junit4:junit4] 2> 15078 T177 C9 P42845 REQ /replication {command=filecontent&file=_0_Lucene41_0.tip&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15078 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10046 (-1429684065853767680)]} 0 0
[junit4:junit4] 2> 15081 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_Asserting_0.pos&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15082 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10048 (1429684065856913408)]} 0 1
[junit4:junit4] 2> 15086 T178 C9 P42845 REQ /replication {command=filecontent&file=_0.fdx&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15087 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10048 (1429684065862156288)]} 0 2
[junit4:junit4] 2> 15090 T179 C9 P42845 REQ /replication {command=filecontent&file=_0.nvm&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=1
[junit4:junit4] 2> 15093 T176 C9 P42845 REQ /replication {command=filecontent&file=_0_Asserting_0.doc&generation=2&wt=filestream&qt=/replication&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 15094 T202 C7 P58139 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 15094 T202 C7 P58139 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130316235614093
[junit4:junit4] 2> 15097 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[46 (-1429684065873690624)]} 0 0
[junit4:junit4] 2> 15100 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[46 (-1429684065876836352)]} 0 0
[junit4:junit4] 2> 15104 T202 C7 P58139 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 15105 T159 C8 P45612 /update {version=2&wt=javabin} {add=[48 (1429684065879982080)]} 0 2
[junit4:junit4] 2> 15105 T202 C7 P58139 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 15105 T202 C7 P58139 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4] 2> 15107 T202 C7 P58139 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index/ new=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index.20130316235614093
[junit4:junit4] 2> 15109 T177 C9 P42845 /update {version=2&wt=javabin} {add=[48 (1429684065885224960)]} 0 1
[junit4:junit4] 2> 15110 T202 C7 P58139 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d),segFN=segments_2,generation=2,filenames=[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_2, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 15110 T202 C7 P58139 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_2, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 15111 T202 C7 P58139 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 15115 T202 C7 P58139 oass.SolrIndexSearcher.<init> Opening Searcher@3ebc368b main
[junit4:junit4] 2> 15116 T201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ebc368b main{StandardDirectoryReader(segments_2:4:nrt _0(4.3):C89/36)}
[junit4:junit4] 2> 15116 T202 C7 P58139 oash.SnapPuller$2.preClose removing old index files MockDirWrapper(org.apache.lucene.store.RAMDirectory@27c5c202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c40d827)
[junit4:junit4] 2> 15117 T202 C7 P58139 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/index
[junit4:junit4] 2> 15117 T202 C7 P58139 oash.SnapPuller$2.postClose removing old index directory MockDirWrapper(org.apache.lucene.store.RAMDirectory@27c5c202 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7c40d827)
[junit4:junit4] 2> 15119 T202 C7 P58139 oasc.RecoveryStrategy.replay Replaying buffered documents. core=collection1
[junit4:junit4] 2> 15119 T210 C7 P58139 oasu.UpdateLog$LogReplayer.doReplay WARNING Starting log replay tlog{file=./org.apache.solr.cloud.RecoveryZkTest-1363452959107/jetty2/tlog/tlog.0000000000000000000 refcount=2} active=true starting pos=4696
[junit4:junit4] 2> 15120 T210 C7 P58139 oasu.DirectUpdateHandler2.commit start commit{flags=2,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 15122 T210 C7 P58139 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d),segFN=segments_2,generation=2,filenames=[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_2, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d),segFN=segments_3,generation=3,filenames=[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_3, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 15122 T210 C7 P58139 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_3, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> 15123 T210 C7 P58139 oass.SolrIndexSearcher.<init> Opening Searcher@31b73442 main
[junit4:junit4] 2> 15123 T210 C7 P58139 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 15124 T201 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31b73442 main{StandardDirectoryReader(segments_2:4:nrt _0(4.3):C89/36)}
[junit4:junit4] 2> 15124 T210 C7 P58139 oasu.UpdateLog$LogReplayer.run WARNING Log replay finished. recoveryInfo=RecoveryInfo{adds=0 deletes=0 deleteByQuery=0 errors=0 positionOfStart=4696}
[junit4:junit4] 2> 15124 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 15125 T202 C7 P58139 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 15125 T202 C7 P58139 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15129 T202 C7 P58139 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 15192 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10049 (1429684065971208192)]} 0 3
[junit4:junit4] 2> 15198 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10049 (1429684065978548224)]} 0 2
[junit4:junit4] 2> 15212 T158 C8 P45612 /update {version=2&wt=javabin} {add=[49 (1429684065993228288)]} 0 1
[junit4:junit4] 2> 15216 T176 C9 P42845 /update {version=2&wt=javabin} {add=[49 (1429684065997422592)]} 0 1
[junit4:junit4] 2> 15248 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10050 (1429684066030977024)]} 0 1
[junit4:junit4] 2> 15252 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10050 (1429684066035171328)]} 0 1
[junit4:junit4] 2> 15265 T157 C8 P45612 /update {version=2&wt=javabin} {add=[50 (1429684066048802816)]} 0 1
[junit4:junit4] 2> 15270 T177 C9 P42845 /update {version=2&wt=javabin} {add=[50 (1429684066054045696)]} 0 2
[junit4:junit4] 2> 15296 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10051 (1429684066081308672)]} 0 1
[junit4:junit4] 2> 15300 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10051 (1429684066085502976)]} 0 1
[junit4:junit4] 2> 15315 T158 C8 P45612 /update {version=2&wt=javabin} {add=[51 (1429684066100183040)]} 0 2
[junit4:junit4] 2> 15319 T176 C9 P42845 /update {version=2&wt=javabin} {add=[51 (1429684066105425920)]} 0 1
[junit4:junit4] 2> 15392 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10052 (1429684066181971968)]} 0 2
[junit4:junit4] 2> 15396 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10052 (1429684066186166272)]} 0 1
[junit4:junit4] 2> 15410 T157 C8 P45612 /update {version=2&wt=javabin} {add=[52 (1429684066200846336)]} 0 1
[junit4:junit4] 2> 15414 T177 C9 P42845 /update {version=2&wt=javabin} {add=[52 (1429684066205040640)]} 0 1
[junit4:junit4] 2> 15496 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10053 (1429684066291023872)]} 0 2
[junit4:junit4] 2> 15500 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10053 (1429684066295218176)]} 0 1
[junit4:junit4] 2> 15513 T159 C8 P45612 /update {version=2&wt=javabin} {add=[53 (1429684066308849664)]} 0 1
[junit4:junit4] 2> 15517 T177 C9 P42845 /update {version=2&wt=javabin} {add=[53 (1429684066313043968)]} 0 1
[junit4:junit4] 2> 15539 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10051 (-1429684066337161216)]} 0 0
[junit4:junit4] 2> 15542 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[10051 (-1429684066340306944)]} 0 0
[junit4:junit4] 2> 15546 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10054 (1429684066343452672)]} 0 1
[junit4:junit4] 2> 15549 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10054 (1429684066347646976)]} 0 1
[junit4:junit4] 2> 15556 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[51 (-1429684066354987008)]} 0 0
[junit4:junit4] 2> 15559 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[51 (-1429684066358132736)]} 0 0
[junit4:junit4] 2> 15563 T159 C8 P45612 /update {version=2&wt=javabin} {add=[54 (1429684066361278464)]} 0 1
[junit4:junit4] 2> 15566 T177 C9 P42845 /update {version=2&wt=javabin} {add=[54 (1429684066365472768)]} 0 1
[junit4:junit4] 2> 15596 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10055 (1429684066395881472)]} 0 2
[junit4:junit4] 2> 15600 T179 C9 P42845 /update {version=2&wt=javabin} {add=[10055 (1429684066400075776)]} 0 1
[junit4:junit4] 2> 15613 T158 C8 P45612 /update {version=2&wt=javabin} {add=[55 (1429684066413707264)]} 0 2
[junit4:junit4] 2> 15617 T178 C9 P42845 /update {version=2&wt=javabin} {add=[55 (1429684066417901568)]} 0 1
[junit4:junit4] 2> 15682 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10054 (-1429684066486059008)]} 0 1
[junit4:junit4] 2> 15686 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[10054 (-1429684066491301888)]} 0 1
[junit4:junit4] 2> 15692 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10056 (1429684066495496192)]} 0 2
[junit4:junit4] 2> 15698 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[54 (-1429684066502836224)]} 0 1
[junit4:junit4] 2> 15698 T164 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15699 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10056 (1429684066502836224)]} 0 3
[junit4:junit4] 2> 15700 T164 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58139",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58139_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 15702 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[54 (-1429684066508079104)]} 0 1
[junit4:junit4] 2> 15708 T158 C8 P45612 /update {version=2&wt=javabin} {add=[56 (1429684066512273408)]} 0 3
[junit4:junit4] 2> 15712 T178 C9 P42845 /update {version=2&wt=javabin} {add=[56 (1429684066517516288)]} 0 1
[junit4:junit4] 2> 15741 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 15741 T199 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 15741 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 15741 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 15763 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10055 (-1429684066572042240)]} 0 1
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=active, collection=collection1, base_url=http://127.0.0.1:58139, shard=shard1, node_name=127.0.0.1:58139_, core=collection1}
[junit4:junit4] 2> 15769 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10055 (-1429684066575187968)]} 0 1
[junit4:junit4] 2> 15770 T176 C9 P42845 /update {version=2&wt=javabin} {delete=[10055 (-1429684066575187968)]} 0 4
[junit4:junit4] 2> 15774 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10057 (1429684066582528000)]} 0 2
[junit4:junit4] 2> 15775 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[55 (-1429684066583576576)]} 0 0
[junit4:junit4] 2> 15780 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[55 (-1429684066586722304)]} 0 1
[junit4:junit4] 2> 15781 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[55 (-1429684066586722304)]} 0 4
[junit4:junit4] 2> 15782 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10057 (1429684066585673728)]} 0 2
[junit4:junit4] 2> 15783 T177 C9 P42845 /update {version=2&wt=javabin} {add=[10057 (1429684066585673728)]} 0 7
[junit4:junit4] 2> 15787 T156 C8 P45612 /update {version=2&wt=javabin} {add=[57 (1429684066595110912)]} 0 1
[junit4:junit4] 2> 15794 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[57 (1429684066600353792)]} 0 1
[junit4:junit4] 2> 15795 T178 C9 P42845 /update {version=2&wt=javabin} {add=[57 (1429684066600353792)]} 0 6
[junit4:junit4] 2> 15807 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10058 (1429684066617131008)]} 0 1
[junit4:junit4] 2> 15819 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10058 (1429684066623422464)]} 0 2
[junit4:junit4] 2> 15820 T159 C8 P45612 /update {version=2&wt=javabin} {add=[58 (1429684066629713920)]} 0 3
[junit4:junit4] 2> 15821 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10058 (1429684066623422464)]} 0 10
[junit4:junit4] 2> 15831 T194 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[58 (1429684066637053952)]} 0 2
[junit4:junit4] 2> 15831 T179 C9 P42845 /update {version=2&wt=javabin} {add=[58 (1429684066637053952)]} 0 7
[junit4:junit4] 2> 15849 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[10056 (-1429684066662219776)]} 0 0
[junit4:junit4] 2> 15857 T195 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[10056 (-1429684066666414080)]} 0 1
[junit4:junit4] 2> 15858 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10056 (-1429684066666414080)]} 0 5
[junit4:junit4] 2> 15859 T141 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 15860 T141 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15864 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[56 (-1429684066677948416)]} 0 1
[junit4:junit4] 2> 15866 T158 C8 P45612 /update {version=2&wt=javabin} {add=[10059 (1429684066677948417)]} 0 3
[junit4:junit4] 2> 15866 T141 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3b84b706
[junit4:junit4] 2> 15871 T192 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {delete=[56 (-1429684066683191296)]} 0 0
[junit4:junit4] 2> 15872 T192 C7 P58139 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3b84b706
[junit4:junit4] 2> 15874 T193 C7 P58139 /update {version=2&wt=javabin&distrib.from=http://127.0.0.1:42845/collection1/&update.distrib=FROMLEADER} {add=[10059 (1429684066684239872)]} 0 1
[junit4:junit4] 2> 15874 T193 C7 P58139 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3b84b706
[junit4:junit4] 2> 15875 T192 C7 P58139 oasc.SolrException.log SEVERE org.apache.solr.common.SolrException: Failed to unregister info bean: Searcher@31b73442 main
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.unregister(JmxMonitoredMap.java:178)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.clear(JmxMonitoredMap.java:116)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.close(SolrCore.java:970)
[junit4:junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
[junit4:junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
[junit4:junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
[junit4:junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:365)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
[junit4:junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
[junit4:junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4] 2> at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> Caused by: javax.management.InstanceNotFoundException: solr/collection1:id=org.apache.solr.search.SolrIndexSearcher,type=Searcher@31b73442 main
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:536)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.unregister(JmxMonitoredMap.java:175)
[junit4:junit4] 2> ... 29 more
[junit4:junit4] 2>
[junit4:junit4] 2> 15875 T141 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=5,adds=5,deletesById=4,deletesByQuery=0,errors=0,cumulative_adds=47,cumulative_deletesById=20,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 15876 T192 C7 P58139 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=5,adds=5,deletesById=4,deletesByQuery=0,errors=0,cumulative_adds=47,cumulative_deletesById=20,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 15876 T141 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 15876 T193 C7 P58139 oasc.SolrException.log SEVERE org.apache.solr.common.SolrException: Failed to unregister info bean: core
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.unregister(JmxMonitoredMap.java:178)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.clear(JmxMonitoredMap.java:116)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.close(SolrCore.java:970)
[junit4:junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:372)
[junit4:junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:135)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
[junit4:junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:275)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1074)
[junit4:junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
[junit4:junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
[junit4:junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
[junit4:junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:365)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:937)
[junit4:junit4] 2> at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:998)
[junit4:junit4] 2> at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:948)
[junit4:junit4] 2> at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
[junit4:junit4] 2> at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
[junit4:junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:628)
[junit4:junit4] 2> at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
[junit4:junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> Caused by: javax.management.InstanceNotFoundException: solr/collection1:id=collection1,type=core
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:536)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.unregister(JmxMonitoredMap.java:175)
[junit4:junit4] 2> ... 29 more
[junit4:junit4] 2>
[junit4:junit4] 2> 15877 T192 C7 P58139 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 15877 T141 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 15877 T193 C7 P58139 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=5,deletesById=4,deletesByQuery=0,errors=0,cumulative_adds=47,cumulative_deletesById=20,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 15877 T141 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 15886 T141 C7 P58139 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d),segFN=segments_3,generation=3,filenames=[_0_Direct_0.doc, _0_Lucene41_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.pos, segments_3, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _0_MockFixedIntBlock_0.tib, _0_Asserting_0.tip, _0_1.del, _0_Direct_0.tim, _0_Direct_0.pos, _0_MockFixedIntBlock_0.doc, _0.si, _0_MockFixedIntBlock_0.frq, _0_Lucene41_0.tim, _0_MockFixedIntBlock_0.pyl, _0_Direct_0.tip, _0_Lucene41_0.doc, _0.fnm, _0.fdt, _0.nvd, _0_Lucene41_0.tip, _0_Asserting_0.pos, _0.fdx, _0.nvm, _0_Asserting_0.doc]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@34a32a11 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19085b9d),segFN=segments_4,generation=4,filenames=[_0_Lucene41_0.pos, _2_Direct_0.doc, _2_MockFixedIntBlock_0.tib, _2.fdt, _0_MockFixedIntBlock_0.tib, _2.fnm, _0_1.del, _0_Direct_0.tim, _2.nvm, _2_MockFixedIntBlock_0.frq, _0.si, _0_Lucene41_0.tim, _2_Lucene41_0.tip, _0_Direct_0.tip, _0.fnm, _0_Lucene41_0.doc, _2_Direct_0.tip, _0_Lucene41_0.tip, _2.fdx, _0_Asserting_0.pos, _2_Direct_0.tim, _0.nvm, _2_Direct_0.pos, _0_Direct_0.doc, _2_MockFixedIntBlock_0.pos, _0_MockFixedIntBlock_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _2.nvd, _2_MockFixedIntBlock_0.tii, _2_Asserting_0.tip, _2.si, _0_Asserting_0.tip, _0_MockFixedIntBlock_0.doc, _0_Direct_0.pos, _2_Lucene41_0.pos, _0_MockFixedIntBlock_0.frq, _2_MockFixedIntBlock_0.doc, _2_Asserting_0.doc, _2_Asserting_0.pos, _0_MockFixedIntBlock_0.pyl, _2_MockFixedIntBlock_0.skp, _2_Asserting_0.tim, _2_Lucene41_0.tim, _2_MockFixedIntBlock_0.pyl, _0.fdt, _2_Lucene41_0.doc, _0.nvd, segments_4, _0.fdx, _0_Asserting_0.doc]
[junit4:junit4] 2> 15886 T141 C7 P58139 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[_0_Lucene41_0.pos, _2_Direct_0.doc, _2_MockFixedIntBlock_0.tib, _2.fdt, _0_MockFixedIntBlock_0.tib, _2.fnm, _0_1.del, _0_Direct_0.tim, _2.nvm, _2_MockFixedIntBlock_0.frq, _0.si, _0_Lucene41_0.tim, _2_Lucene41_0.tip, _0_Direct_0.tip, _0.fnm, _0_Lucene41_0.doc, _2_Direct_0.tip, _0_Lucene41_0.tip, _2.fdx, _0_Asserting_0.pos, _2_Direct_0.tim, _0.nvm, _2_Direct_0.pos, _0_Direct_0.doc, _2_MockFixedIntBlock_0.pos, _0_MockFixedIntBlock_0.pos, _0_Asserting_0.tim, _0_MockFixedIntBlock_0.tii, _0_MockFixedIntBlock_0.skp, _2.nvd, _2_MockFixedIntBlock_0.tii, _2_Asserting_0.tip, _2.si, _0_Asserting_0.tip, _0_MockFixedIntBlock_0.doc, _0_Direct_0.pos, _2_Lucene41_0.pos, _0_MockFixedIntBlock_0.frq, _2_MockFixedIntBlock_0.doc, _2_Asserting_0.doc, _2_Asserting_0.pos, _0_MockFixedIntBlock_0.pyl, _2_MockFixedIntBlock_0.skp, _2_Asserting_0.tim, _2_Lucene41_0.tim, _2_MockFixedIntBlock_0.pyl, _0.fdt, _2_Lucene41_0.doc, _0.nvd, segments_4, _0.fdx, _0_Asserting_0.doc]
[junit4:junit4] 2> 15888 T193 C7 P58139 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 15888 T141 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 15889 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[56 (-1429684066683191296)]} 0 21
[junit4:junit4] 2> 15889 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10059 (1429684066684239872)]} 0 19
[junit4:junit4] 2> 15890 T146 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d742119880008
[junit4:junit4] 2> 15890 T163 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 15891 T141 oaz.ZooKeeper.close Session: 0x13d742119880008 closed
[junit4:junit4] 2> 15891 T143 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:57286 which had sessionid 0x13d742119880008
[junit4:junit4] 2> 15892 T170 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 15892 T183 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 15892 T170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 15892 T183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 15893 T163 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 15893 T159 C8 P45612 /update {version=2&wt=javabin} {add=[59 (1429684066707308544)]} 0 1
[junit4:junit4] 2> 15897 T179 C9 P42845 /update {version=2&wt=javabin} {add=[59 (1429684066711502848)]} 0 1
[junit4:junit4] 2> 15913 T141 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 15943 T157 C8 P45612 /update {version=2&wt=javabin} {add=[10060 (1429684066758688768)]} 0 3
[junit4:junit4] 2> 15949 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10060 (1429684066764980224)]} 0 2
[junit4:junit4] 2> 15950 T158 C8 P45612 /update {version=2&wt=javabin} {add=[60 (1429684066766028800)]} 0 3
[junit4:junit4] 2> 15956 T176 C9 P42845 /update {version=2&wt=javabin} {add=[60 (1429684066772320256)]} 0 2
[junit4:junit4] 2> 16044 T159 C8 P45612 /update {version=2&wt=javabin} {delete=[10057 (-1429684066866692096)]} 0 0
[junit4:junit4] 2> 16047 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[10057 (-1429684066869837824)]} 0 0
[junit4:junit4] 2> 16050 T157 C8 P45612 /update {version=2&wt=javabin} {delete=[57 (-1429684066872983552)]} 0 0
[junit4:junit4] 2> 16051 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10061 (1429684066872983553)]} 0 1
[junit4:junit4] 2> 16053 T179 C9 P42845 /update {version=2&wt=javabin} {delete=[57 (-1429684066875080704)]} 0 1
[junit4:junit4] 2> 16054 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10061 (1429684066876129280)]} 0 1
[junit4:junit4] 2> 16056 T158 C8 P45612 /update {version=2&wt=javabin} {add=[61 (1429684066878226432)]} 0 1
[junit4:junit4] 2> 16060 T177 C9 P42845 /update {version=2&wt=javabin} {add=[61 (1429684066882420736)]} 0 2
[junit4:junit4] 2> 16063 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10062 (1429684066885566464)]} 0 1
[junit4:junit4] 2> 16068 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10062 (1429684066890809344)]} 0 2
[junit4:junit4] 2> 16069 T157 C8 P45612 /update {version=2&wt=javabin} {add=[62 (1429684066891857920)]} 0 2
[junit4:junit4] 2> 16073 T179 C9 P42845 /update {version=2&wt=javabin} {add=[62 (1429684066896052224)]} 0 1
[junit4:junit4] 2> 16081 T156 C8 P45612 /update {version=2&wt=javabin} {delete=[10058 (-1429684066905489408)]} 0 0
[junit4:junit4] 2> 16084 T178 C9 P42845 /update {version=2&wt=javabin} {delete=[10058 (-1429684066908635136)]} 0 0
[junit4:junit4] 2> 16086 T158 C8 P45612 /update {version=2&wt=javabin} {delete=[58 (-1429684066910732288)]} 0 0
[junit4:junit4] 2> 16088 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10063 (1429684066911780864)]} 0 2
[junit4:junit4] 2> 16089 T177 C9 P42845 /update {version=2&wt=javabin} {delete=[58 (-1429684066913878016)]} 0 0
[junit4:junit4] 2> 16091 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10063 (1429684066914926592)]} 0 1
[junit4:junit4] 2> 16093 T157 C8 P45612 /update {version=2&wt=javabin} {add=[63 (1429684066917023744)]} 0 1
[junit4:junit4] 2> 16096 T179 C9 P42845 /update {version=2&wt=javabin} {add=[63 (1429684066920169472)]} 0 1
[junit4:junit4] 2> 16153 T156 C8 P45612 /update {version=2&wt=javabin} {add=[10064 (1429684066978889728)]} 0 3
[junit4:junit4] 2> 16158 T158 C8 P45612 /update {version=2&wt=javabin} {add=[64 (1429684066983084032)]} 0 3
[junit4:junit4] 2> 16159 T178 C9 P42845 /update {version=2&wt=javabin} {add=[10064 (1429684066986229760)]} 0 2
[junit4:junit4] 2> 16163 T177 C9 P42845 /update {version=2&wt=javabin} {add=[64 (1429684066990424064)]} 0 2
[junit4:junit4] 2> 16167 T141 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 16170 T141 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58139
[junit4:junit4] 2> 16170 T141 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 16171 T141 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 16171 T141 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237
[junit4:junit4] 2> 16171 T141 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/solr.xml
[junit4:junit4] 2> 16172 T141 oasc.CoreContainer.<init> New CoreContainer 609850803
[junit4:junit4] 2> 16172 T141 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/'
[junit4:junit4] 2> 16172 T141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.RecoveryZkTest-jetty2-1363452967237/'
[junit4:junit4] 2> 16179 T159 C8 P45612 /update {version=2&wt=javabin} {add=[10065 (1429684067007201280)]} 0 1
[junit4:junit4] 2> 16183 T176 C9 P42845 /update {version=2&wt=javabin} {add=[10065 (1429684067011395584)]} 0 1
[junit4:junit4] 2> 16184 T157 C8 P45612 /update {version=2&wt=javabin} {add=[65 (1429684067012444160)]} 0 1
[junit4:junit4] 2> 16188 T179 C9 P42845 /update {version=2&wt=javabin} {add=[65 (1429684067016638464)]} 0 1
[junit4:junit4] 2> 16195 T141 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 16195 T141 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 16195 T141 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 16196 T141 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 16196 T141 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 16196 T141 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 16197 T1
[...truncated too long message...]
nit4] 1> "core":"unloadcollection2"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:44544_qp_qtp%2Fgm (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:48307_qp_qtp%2Fgm (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:53930_qp_qtp%2Fgm (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:33407_qp_qtp%2Fgm (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:51245_qp_qtp%2Fgm (0)
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UnloadDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=DDA8E93F6FE5B250 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ru_RU -Dtests.timezone=America/Menominee -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 138s J0 | UnloadDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Still found shard
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DDA8E93F6FE5B250:5C4E672718BAD26C]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 138074 T1329 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 138088 T1328 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 138724 T1332 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> 139181 T1411 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 139181 T1411 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 139181 T1411 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {intDefault=PostingsFormat(name=MockRandom), other_tl1=PostingsFormat(name=MockRandom), range_facet_si=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=MockSep), multiDefault=PostingsFormat(name=MockSep), timestamp=PostingsFormat(name=MockSep), id=Pulsing41(freqCutoff=7 minBlockSize=82 maxBlockSize=245), a_si=PostingsFormat(name=MockRandom), text=MockFixedIntBlock(blockSize=1941), a_t=PostingsFormat(name=MockSep), range_facet_sl=Pulsing41(freqCutoff=7 minBlockSize=82 maxBlockSize=245)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ru_RU, timezone=America/Menominee
[junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=1,free=143096088,total=278724608
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SpatialFilterTest, SoftAutoCommitTest, DateFieldTest, ZkControllerTest, TestRemoteStreaming, PeerSyncTest, RecoveryZkTest, ExternalFileFieldSortTest, SimpleFacetsTest, TestCharFilters, LegacyHTMLStripCharFilterTest, QueryEqualityTest, TestFuzzyAnalyzedSuggestions, DocumentBuilderTest, TestRandomDVFaceting, ChaosMonkeyNothingIsSafeTest, TestSolrQueryParser, ClusterStateUpdateTest, TestStressVersions, TestConfig, AliasIntegrationTest, TestShardHandlerFactory, TestTrie, DistributedQueryElevationComponentTest, TestPropInjectDefaults, TestReplicationHandler, QueryElevationComponentTest, QueryParsingTest, TestPseudoReturnFields, DirectSolrConnectionTest, TestBadConfig, TestFaceting, SolrCoreTest, UUIDFieldTest, UnloadDistributedZkTest]
[junit4:junit4] Completed on J0 in 139.38s, 1 test, 1 failure <<< FAILURES!
[...truncated 551 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1213: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:877: There were test failures: 273 suites, 1147 tests, 2 suite-level errors, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 39 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 64bit/jdk1.8.0-ea-b79 -XX:+UseParallelGC
Email was triggered for: Failure
Sending email for trigger: Failure