You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/06/27 22:53:43 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4094 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4094/
8 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest: 1) Thread[id=1653, name=IPC Parameter Sending Thread #2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardSplitTest:
1) Thread[id=1653, name=IPC Parameter Sending Thread #2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardSplitTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1653, name=IPC Parameter Sending Thread #2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1653, name=IPC Parameter Sending Thread #2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.store.hdfs.HdfsDirectoryTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
FAILED: org.apache.solr.TestSolrCoreProperties.testSimple
Error Message:
System properties invariant violated. Missing keys: tests.shardhandler.randomSeed=6269827861127365263
Stack Trace:
java.lang.AssertionError: System properties invariant violated.
Missing keys:
tests.shardhandler.randomSeed=6269827861127365263
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D:EF5BBA9F637C74DC]:0)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:66)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D:6963F8CE3DF5AE38]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.junit.Assert.assertNotNull(Assert.java:537)
at org.apache.solr.cloud.ClusterStateUpdateTest.testCoreRegistration(ClusterStateUpdateTest.java:201)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.BasicHdfsTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
Build Log:
[...truncated 9522 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.hdfs.BasicHdfsTest
[junit4:junit4] 2> 64205 T196 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /waxvc/l
[junit4:junit4] 2> 66037 T196 oahu.NativeCodeLoader.<clinit> WARN Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4:junit4] 2> 66587 T196 oahc.Configuration.warnOnceIfDeprecated WARN hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
[junit4:junit4] 2> 67814 T196 oahmi.MetricsConfig.loadFirst WARN Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4:junit4] 2> 70426 T196 oml.Slf4jLog.info Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4:junit4] 2> 70819 T196 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 70978 T196 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/hdfs to ./Jetty_localhost_61377_hdfs____.1gkgu/webapp
[junit4:junit4] 2> 71178 T196 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 71430 T196 oml.Slf4jLog.info Started SelectChannelConnector@localhost:61377
[junit4:junit4] 2> 72290 T196 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 72356 T196 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/datanode to ./Jetty_localhost_61402_datanode____.2kug3i/webapp
[junit4:junit4] 2> 72443 T196 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 72516 T196 oml.Slf4jLog.info Started SelectChannelConnector@localhost:61402
[junit4:junit4] 2> 73473 T196 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 74745 T196 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/datanode to ./Jetty_localhost_61430_datanode____.hygs9v/webapp
[junit4:junit4] 2> 75652 T196 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 76657 T196 oml.Slf4jLog.info Started SelectChannelConnector@localhost:61430
[junit4:junit4] 1> Formatting using clusterid: testClusterID
[junit4:junit4] 2> 81122 T221 oahhsb.BlockManager.processReport BLOCK* processReport: from DatanodeRegistration(140.211.11.196, storageID=DS-1900384430-140.211.11.196-61429-1372349882713, infoPort=61430, ipcPort=61431, storageInfo=lv=-40;cid=testClusterID;nsid=1519744745;c=0), blocks: 0, processing time: 25 msecs
[junit4:junit4] 2> 81131 T222 oahhsb.BlockManager.processReport BLOCK* processReport: from DatanodeRegistration(140.211.11.196, storageID=DS-1367996326-140.211.11.196-61401-1372349882668, infoPort=61402, ipcPort=61406, storageInfo=lv=-40;cid=testClusterID;nsid=1519744745;c=0), blocks: 0, processing time: 0 msecs
[junit4:junit4] 2> 81142 T196 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicHdfsTest-1372349883885
[junit4:junit4] 2> 81160 T196 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 81176 T305 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 81479 T196 oasc.ZkTestServer.run start zk server on port:61457
[junit4:junit4] 2> 81746 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 81791 T306 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4] 2> 83879 T309 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d7e16fc name:ZooKeeperConnection Watcher:127.0.0.1:61457 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 83880 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 83886 T196 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 83997 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 84000 T313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e8c9b47 name:ZooKeeperConnection Watcher:127.0.0.1:61457/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 84000 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 84014 T196 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 84022 T196 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 84028 T196 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 84034 T196 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 84045 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 84082 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 84099 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 84100 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 84212 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 84213 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 84219 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 84220 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 84225 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 84227 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 84232 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 84233 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 84239 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 84240 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 84246 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 84247 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 84253 T196 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 84254 T196 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 84545 T196 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 84549 T196 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61463
[junit4:junit4] 2> 84550 T196 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 84551 T196 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 84552 T196 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002
[junit4:junit4] 2> 84553 T196 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/solr.xml
[junit4:junit4] 2> 84553 T196 oasc.CoreContainer.<init> New CoreContainer 2011077225
[junit4:junit4] 2> 84554 T196 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/'
[junit4:junit4] 2> 84555 T196 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/'
[junit4:junit4] 2> 84706 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002
[junit4:junit4] 2> 84708 T196 oasc.SolrCoreDiscoverer.addCore Discovered properties file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/conf/core.properties, adding to cores
[junit4:junit4] 2> 84714 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/collection1
[junit4:junit4] 2> 84715 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/collection1/lib
[junit4:junit4] 2> 84717 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/collection1/lib/classes
[junit4:junit4] 2> 84720 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/collection1/conf
[junit4:junit4] 2> 84766 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-controljetty-1372349887002/collection1/conf/xslt
[junit4:junit4] 2> 84885 T196 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 84886 T196 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 84887 T196 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 84888 T196 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 84888 T196 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 84889 T196 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 84890 T196 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 84890 T196 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 84891 T196 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 84892 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 84915 T196 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 84916 T196 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61457/solr
[junit4:junit4] 2> 84953 T196 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 84954 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 84958 T324 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b8c47a2 name:ZooKeeperConnection Watcher:127.0.0.1:61457 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 84959 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 85013 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 85035 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 85038 T326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7396d076 name:ZooKeeperConnection Watcher:127.0.0.1:61457/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 85039 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 85054 T196 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 85074 T196 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 85097 T196 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 85101 T196 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61463_waxvc%2Fl
[junit4:junit4] 2> 85111 T196 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61463_waxvc%2Fl
[junit4:junit4] 2> 85117 T196 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 85151 T196 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 85156 T196 oasc.Overseer.start Overseer (id=89938322024759299-127.0.0.1:61463_waxvc%2Fl-n_0000000000) starting
[junit4:junit4] 2> 85191 T196 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 85217 T328 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 85218 T196 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 85222 T196 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 85225 T196 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 85260 T327 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 85278 T196 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 85279 T196 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 85280 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 85301 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 85309 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 85312 T331 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53d48c1b name:ZooKeeperConnection Watcher:127.0.0.1:61457/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 85312 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 85314 T196 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 85321 T196 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 85611 T196 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 85616 T196 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:61467
[junit4:junit4] 2> 85617 T196 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 85618 T196 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 85618 T196 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063
[junit4:junit4] 2> 85619 T196 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/solr.xml
[junit4:junit4] 2> 85620 T196 oasc.CoreContainer.<init> New CoreContainer 599770809
[junit4:junit4] 2> 85620 T196 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/'
[junit4:junit4] 2> 85621 T196 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/'
[junit4:junit4] 2> 85762 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063
[junit4:junit4] 2> 85767 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/collection1
[junit4:junit4] 2> 85768 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/collection1/conf
[junit4:junit4] 2> 85867 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/collection1/conf/xslt
[junit4:junit4] 2> 85927 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/collection1/lib
[junit4:junit4] 2> 85929 T196 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/collection1/lib/classes
[junit4:junit4] 2> 85932 T196 oasc.SolrCoreDiscoverer.addCore Discovered properties file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/conf/core.properties, adding to cores
[junit4:junit4] 2> 85942 T196 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 85943 T196 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 85943 T196 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 85944 T196 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 85944 T196 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 85945 T196 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 85946 T196 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 85946 T196 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 85947 T196 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 85948 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 85968 T196 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 85969 T196 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:61457/solr
[junit4:junit4] 2> 85969 T196 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 85971 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86010 T342 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15407a26 name:ZooKeeperConnection Watcher:127.0.0.1:61457 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86011 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86015 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 86030 T196 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 86075 T344 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c2e6e46 name:ZooKeeperConnection Watcher:127.0.0.1:61457/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 86075 T196 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 86110 T196 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 87116 T196 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61467_waxvc%2Fl
[junit4:junit4] 2> 87118 T196 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61467_waxvc%2Fl
[junit4:junit4] 2> 87130 T331 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87131 T326 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87131 T344 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 87146 T196 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 87147 T196 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 87147 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 87166 T196 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 87188 T334 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=1&maxShardsPerNode=1&name=delete_data_dir&replicationFactor=1&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 87192 T326 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 87200 T328 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "maxShardsPerNode":"1",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"delete_data_dir",
[junit4:junit4] 2> "replicationFactor":"1"}
[junit4:junit4] 2> 87200 T328 oasc.OverseerCollectionProcessor.createCollection Creating shard delete_data_dir_shard1_replica1 as part of slice shard1 of collection delete_data_dir on 127.0.0.1:61467_waxvc%2Fl
[junit4:junit4] 2> 87217 T335 oasc.ZkController.publish publishing core=delete_data_dir_shard1_replica1 state=down
[junit4:junit4] 2> 87220 T335 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 88270 T327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 88277 T327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"delete_data_dir_shard1_replica1",
[junit4:junit4] 2> "collection":"delete_data_dir",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61467_waxvc%2Fl",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61467/waxvc/l"}
[junit4:junit4] 2> 88278 T327 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delete_data_dir with numShards 1
[junit4:junit4] 2> 88330 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 88330 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 88330 T326 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 89222 T335 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delete_data_dir_shard1_replica1
[junit4:junit4] 2> 89223 T335 oasc.CoreContainer.create Creating SolrCore 'delete_data_dir_shard1_replica1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/delete_data_dir_shard1_replica1
[junit4:junit4] 2> 89223 T335 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delete_data_dir
[junit4:junit4] 2> 89225 T335 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:delete_data_dir
[junit4:junit4] 2> 89226 T335 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 89228 T335 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 89228 T335 oascc.SolrZkClient.makePath makePath: /collections/delete_data_dir
[junit4:junit4] 2> 89234 T335 oasc.ZkController.readConfigName Load collection config from:/collections/delete_data_dir
[junit4:junit4] 2> 89238 T335 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/delete_data_dir_shard1_replica1/'
[junit4:junit4] 2> 89359 T335 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 89451 T335 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 89553 T335 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 89566 T335 oass.IndexSchema.readSchema [delete_data_dir_shard1_replica1] Schema name=test
[junit4:junit4] 2> 90383 T335 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 90396 T335 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 90400 T335 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 90430 T335 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 90437 T335 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 90443 T335 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 90447 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 90448 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 90449 T335 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 90452 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 90453 T335 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 90454 T335 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.HdfsDirectoryFactory
[junit4:junit4] 2> 90480 T335 oasc.HdfsDirectoryFactory.init Solr Kerberos Authentication disabled
[junit4:junit4] 2> 90481 T335 oasc.SolrCore.<init> [delete_data_dir_shard1_replica1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.hdfs.BasicHdfsTest-jetty1-1372349888063/delete_data_dir_shard1_replica1/, dataDir=hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/
[junit4:junit4] 2> 90481 T335 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6134cd50
[junit4:junit4] 2> 90483 T335 oasc.HdfsDirectoryFactory.create creating directory factory for path hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data
[junit4:junit4] 2> 90795 T335 oasc.CachingDirectoryFactory.get return new directory for hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data
[junit4:junit4] 2> 90799 T335 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index/
[junit4:junit4] 2> 90828 T335 oasc.SolrCore.initIndex WARN [delete_data_dir_shard1_replica1] Solr index directory 'hdfs:/lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 90828 T335 oasc.HdfsDirectoryFactory.create creating directory factory for path hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index
[junit4:junit4] 2> 90829 T335 oasc.HdfsDirectoryFactory.create Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4:junit4] 2> 90830 T335 oasc.HdfsDirectoryFactory.create Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4:junit4] 2> 90837 T335 oassb.BufferStore.init Initializing the 1024 buffers with [8192] buffers.
[junit4:junit4] 2> 90913 T335 oassb.BufferStore.init Initializing the 8192 buffers with [8192] buffers.
[junit4:junit4] 2> 91215 T335 oasc.CachingDirectoryFactory.get return new directory for hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index
[junit4:junit4] 2> 92120 T217 oahhsb.BlockManager.logAddStoredBlock BLOCK* addStoredBlock: blockMap updated: 140.211.11.196:61401 is added to blk_697615765490055834_1003{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[140.211.11.196:61429|RBW], ReplicaUnderConstruction[140.211.11.196:61401|RBW]]} size 0
[junit4:junit4] 2> 92152 T219 oahhsb.BlockManager.logAddStoredBlock BLOCK* addStoredBlock: blockMap updated: 140.211.11.196:61429 is added to blk_697615765490055834_1003 size 45
[junit4:junit4] 2> 92160 T335 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.solr.store.hdfs.HdfsDirectory@76961f1f lockFactory=org.apache.solr.store.hdfs.HdfsLockFactory@1087660; maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 92160 T335 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 92237 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 92238 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 92240 T335 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 92241 T335 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 92242 T335 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 92243 T335 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 92244 T335 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 92245 T335 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 92246 T335 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 92263 T335 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 92311 T335 oass.SolrIndexSearcher.<init> Opening Searcher@7184015a main
[junit4:junit4] 2> 92373 T335 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 92374 T335 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 92379 T356 oasc.SolrCore.registerSearcher [delete_data_dir_shard1_replica1] Registered new searcher Searcher@7184015a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 92385 T335 oasc.CoreContainer.registerCore registering core: delete_data_dir_shard1_replica1
[junit4:junit4] 2> 92385 T335 oasc.ZkController.register Register replica - core:delete_data_dir_shard1_replica1 address:http://127.0.0.1:61467/waxvc/l collection:delete_data_dir shard:shard1
[junit4:junit4] 2> 92398 T335 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 92420 T335 oascc.SolrZkClient.makePath makePath: /collections/delete_data_dir/leader_elect/shard1/election
[junit4:junit4] 2> 92441 T335 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 92453 T335 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 92454 T335 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 92455 T335 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1/
[junit4:junit4] 2> 92455 T335 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 92456 T335 oasc.SyncStrategy.syncToMe http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1/ has no replicas
[junit4:junit4] 2> 92456 T335 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1/ shard1
[junit4:junit4] 2> 92457 T335 oascc.SolrZkClient.makePath makePath: /collections/delete_data_dir/leaders/shard1
[junit4:junit4] 2> 92845 T327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 92866 T326 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 92866 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 92866 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 92876 T335 oasc.ZkController.register We are http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1/ and leader is http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1/
[junit4:junit4] 2> 92877 T335 oasc.ZkController.register No LogReplay needed for core=delete_data_dir_shard1_replica1 baseURL=http://127.0.0.1:61467/waxvc/l
[junit4:junit4] 2> 92877 T335 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 93014 T335 oasc.ZkController.publish publishing core=delete_data_dir_shard1_replica1 state=active
[junit4:junit4] 2> 93018 T335 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 93020 T335 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=1&shard=shard1&name=delete_data_dir_shard1_replica1&action=CREATE&collection=delete_data_dir&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=5810
[junit4:junit4] 2> 93022 T328 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: delete_data_dir
[junit4:junit4] 2> 93032 T344 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 93035 T326 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 93035 T328 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=5810},core=delete_data_dir_shard1_replica1}}}
[junit4:junit4] 2> 93040 T334 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=1&maxShardsPerNode=1&name=delete_data_dir&replicationFactor=1&action=CREATE&wt=javabin&version=2} status=0 QTime=5859
[junit4:junit4] 2> 93041 T196 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: delete_data_dir failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 93043 T196 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 94046 T196 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 94517 T327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 94519 T327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"delete_data_dir_shard1_replica1",
[junit4:junit4] 2> "collection":"delete_data_dir",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:61467_waxvc%2Fl",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:61467/waxvc/l"}
[junit4:junit4] 2> 94533 T326 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 94533 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 94533 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 95049 T196 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 95051 T196 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: delete_data_dir
[junit4:junit4] 2> 95052 T196 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C27 name=delete_data_dir_shard1_replica1 org.apache.solr.core.SolrCore@42871e10 url=http://127.0.0.1:61467/waxvc/l/delete_data_dir_shard1_replica1 node=127.0.0.1:61467_waxvc%2Fl C27_STATE=coll:delete_data_dir core:delete_data_dir_shard1_replica1 props:{state=active, core=delete_data_dir_shard1_replica1, node_name=127.0.0.1:61467_waxvc%2Fl, base_url=http://127.0.0.1:61467/waxvc/l, leader=true}
[junit4:junit4] 2> 95213 T336 C27 P61467 oasc.SolrCore.execute [delete_data_dir_shard1_replica1] webapp=/waxvc/l path=/admin/system params={wt=javabin&qt=/admin/system&version=2} status=0 QTime=134
[junit4:junit4] 2> 95223 T316 oasha.CollectionsHandler.handleDeleteAction Deleting Collection : name=delete_data_dir&action=DELETE&wt=javabin&version=2
[junit4:junit4] 2> 95253 T326 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 95256 T328 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"deletecollection",
[junit4:junit4] 2> "name":"delete_data_dir"}
[junit4:junit4] 2> 95256 T328 oasc.OverseerCollectionProcessor.collectionCmd Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4:junit4] 2> 95257 T328 oasc.OverseerCollectionProcessor.collectionCmd Collection Admin sending CoreAdmin cmd to 127.0.0.1:61467/waxvc/l params:action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true&core=delete_data_dir_shard1_replica1&qt=%2Fadmin%2Fcores
[junit4:junit4] 2> 95261 T335 oasha.CoreAdminHandler.handleUnloadAction Unregistering core delete_data_dir_shard1_replica1 from cloudstate.
[junit4:junit4] 2> 95273 T335 oasc.SolrCore.close [delete_data_dir_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@42871e10
[junit4:junit4] 2> 95284 T335 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 95285 T335 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 95285 T335 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 95286 T335 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 95287 T335 oasc.SolrCore.closeSearcher [delete_data_dir_shard1_replica1] Closing main searcher on request.
[junit4:junit4] 2> 95288 T335 oasc.CachingDirectoryFactory.close Closing HdfsDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 95288 T335 oasc.CachingDirectoryFactory.closeCacheValue looking to close hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data [CachedDir<<refCount=0;path=hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data;done=false>>]
[junit4:junit4] 2> 95289 T335 oasc.CachingDirectoryFactory.closeCacheValue looking to close hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data []
[junit4:junit4] 2> 95289 T335 oasc.CachingDirectoryFactory.closeCacheValue looking to close hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index [CachedDir<<refCount=0;path=hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data;done=false>>, CachedDir<<refCount=0;path=hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index;done=false>>]
[junit4:junit4] 2> 95290 T335 oasc.CachingDirectoryFactory.close Closing directory: hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data
[junit4:junit4] 2> 95290 T335 oassh.HdfsDirectory.close Closing hdfs directory hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data
[junit4:junit4] 2> 95291 T335 oasc.CachingDirectoryFactory.close Closing directory: hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index
[junit4:junit4] 2> 95298 T335 oassh.HdfsDirectory.close Closing hdfs directory hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data/index
[junit4:junit4] 2> 95298 T335 oasc.CachingDirectoryFactory.close Removing directory after core close: hdfs://lucene.zones.apache.org:61375/solr/delete_data_dir/core_node1/data
[junit4:junit4] 2> 95324 T219 oahhsb.BlockManager.addToInvalidates BLOCK* addToInvalidates: blk_697615765490055834_1003 140.211.11.196:61401 140.211.11.196:61429
[junit4:junit4] 2> 95338 T335 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&action=UNLOAD&core=delete_data_dir_shard1_replica1&wt=javabin&qt=/admin/cores&deleteDataDir=true&version=2} status=0 QTime=77
[junit4:junit4] 2> 96041 T327 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 96099 T326 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 96099 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 96099 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 96257 T326 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000002 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 96260 T326 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 96260 T328 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000002 complete, response:{success={127.0.0.1:61467_waxvc%2Fl={responseHeader={status=0,QTime=77}}}}
[junit4:junit4] 2> 96275 T316 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={name=delete_data_dir&action=DELETE&wt=javabin&version=2} status=0 QTime=1052
[junit4:junit4] 2> 96323 T196 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 61463
[junit4:junit4] 2> 96324 T196 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=2011077225
[junit4:junit4] 2> 97333 T327 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89938322024759299-127.0.0.1:61463_waxvc%2Fl-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 97337 T196 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/waxvc/l,null}
[junit4:junit4] 2> 97337 T331 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 97337 T344 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 97338 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 97339 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 97343 T344 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 97347 T344 oasc.Overseer.start Overseer (id=89938322024759302-127.0.0.1:61467_waxvc%2Fl-n_0000000001) starting
[junit4:junit4] 2> 97363 T358 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 97366 T357 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 97390 T196 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 61467
[junit4:junit4] 2> 97390 T196 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=599770809
[junit4:junit4] 2> 98393 T357 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89938322024759302-127.0.0.1:61467_waxvc%2Fl-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 98397 T196 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/waxvc/l,null}
[junit4:junit4] 2> 98397 T331 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4:junit4] 2> 98398 T331 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4:junit4] 2> 98452 T196 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 61463
[junit4:junit4] 2> 98457 T196 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 98461 T196 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:61457 61457
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 98597 T196 oahhsd.DirectoryScanner.shutdown WARN DirectoryScanner: shutdown has been called
[junit4:junit4] 2> 98607 T196 oml.Slf4jLog.info Stopped SelectChannelConnector@localhost:0
[junit4:junit4] 2> 98636 T326 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 98636 T326 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 98637 T326 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 99038 T266 oahhsd.BPServiceActor.run WARN Ending block pool service for: Block pool BP-3923770-140.211.11.196-1372349869653 (storage id DS-1900384430-140.211.11.196-61429-1372349882713) service to lucene.zones.apache.org/140.211.11.196:61375
[junit4:junit4] 2> 99696 T344 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 99696 T344 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 99697 T344 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 7200027 T195 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.cloud.hdfs.BasicHdfsTest
[junit4:junit4] 2> ==== jstack at approximately timeout time ====
[junit4:junit4] 2> "refreshUsed-/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test_1372349866949/hdfsBaseDir/data/data2/current/BP-3923770-140.211.11.196-1372349869653" ID=303 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:85)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "refreshUsed-/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test_1372349866949/hdfsBaseDir/data/data1/current/BP-3923770-140.211.11.196-1372349869653" ID=300 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:85)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "java.util.concurrent.ThreadPoolExecutor$Worker@62ea5315[State = -1, empty queue]" ID=296 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6c3cf593
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6c3cf593
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
[junit4:junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
[junit4:junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Thread-195" ID=290 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:78)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Parameter Sending Thread #0" ID=278 TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@161ebd69
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.concurrent.SynchronousQueue$TransferStack@161ebd69
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Client (2002160184) connection to lucene.zones.apache.org/140.211.11.196:61375 from hudson" ID=277 TIMED_WAITING on org.apache.hadoop.ipc.Client$Connection@22ea2abf
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.apache.hadoop.ipc.Client$Connection@22ea2abf
[junit4:junit4] 2> at org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:799)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:843)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@410666dd" ID=259 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4] 2> - locked java.lang.Object@3c2543d7
[junit4:junit4] 2> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:99)
[junit4:junit4] 2> - locked java.lang.Object@3288f71a
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:136)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 9 on 61406" ID=252 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 8 on 61406" ID=251 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 7 on 61406" ID=250 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 6 on 61406" ID=249 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 5 on 61406" ID=248 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 4 on 61406" ID=247 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 3 on 61406" ID=246 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 2 on 61406" ID=245 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 1 on 61406" ID=244 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 0 on 61406" ID=243 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1d6123b0
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server listener on 61406" ID=239 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@514e369f
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@65e42a6c
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@6117755b
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener.run(Server.java:627)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server Responder" ID=241 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@e838744
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@4b387a81
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@1da781d1
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:811)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.run(Server.java:794)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@69945924" ID=235 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4] 2> - locked java.lang.Object@603bcf8c
[junit4:junit4] 2> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:99)
[junit4:junit4] 2> - locked java.lang.Object@76b721af
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:136)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "DataNode: [file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test_1372349866949/hdfsBaseDir/data/data1,file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test_1372349866949/hdfsBaseDir/data/data2] heartbeating to lucene.zones.apache.org/140.211.11.196:61375" ID=242 TIMED_WAITING on java.util.HashMap@7104a50c
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.HashMap@7104a50c
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:573)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:676)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Socket Reader #1 for port 61406" ID=240 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@39a9d4bc
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@5fd19a10
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@7343adf
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:514)
[junit4:junit4] 2> - locked org.apache.hadoop.ipc.Server$Listener$Reader@6de8abf8
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:500)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer-1" ID=238 TIMED_WAITING on java.util.TaskQueue@5b410853
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@5b410853
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "482476724@qtp-1389712043-1 - Acceptor0 SelectChannelConnector@localhost:61402" ID=237 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@44de87ee
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@4a4cf7cd
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@61bdb3db
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
[junit4:junit4] 2> at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
[junit4:junit4] 2> at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[junit4:junit4] 2>
[junit4:junit4] 2> "943184639@qtp-1389712043-0" ID=236 TIMED_WAITING on org.mortbay.thread.QueuedThreadPool$PoolThread@3837daff
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.mortbay.thread.QueuedThreadPool$PoolThread@3837daff
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@6abdc70f" ID=227 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:3492)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@27a64b69" ID=226 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:419)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 9 on 61375" ID=225 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 8 on 61375" ID=224 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 7 on 61375" ID=223 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 6 on 61375" ID=222 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 5 on 61375" ID=221 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 4 on 61375" ID=220 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 3 on 61375" ID=219 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 2 on 61375" ID=218 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 1 on 61375" ID=217 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 0 on 61375" ID=216 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@a89aff3
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server listener on 61375" ID=204 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@14433bd1
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@4a0c20bb
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@ae4a082
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener.run(Server.java:627)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server Responder" ID=206 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@5bb35db2
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@75f16e62
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@30bab202
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:811)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.run(Server.java:794)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer-0" ID=215 TIMED_WAITING on java.util.TaskQueue@557e7fec
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@557e7fec
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "1358941483@qtp-329635714-1 - Acceptor0 SelectChannelConnector@localhost:61377" ID=214 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@202aecb4
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@24bcd19e
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@7c28388a
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
[junit4:junit4] 2> at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
[junit4:junit4] 2> at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[junit4:junit4] 2>
[junit4:junit4] 2> "366574437@qtp-329635714-0" ID=213 TIMED_WAITING on org.mortbay.thread.QueuedThreadPool$PoolThread@15d97b65
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.mortbay.thread.QueuedThreadPool$PoolThread@15d97b65
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@58f16635" ID=202 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3079)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@506d4f31" ID=203 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:295)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@5b657f4" ID=211 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor.run(DecommissionManager.java:76)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor@124efc2" ID=210 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor.run(PendingReplicationBlocks.java:211)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Socket Reader #1 for port 61375" ID=205 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@725d5165
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@4c8af88b
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@966f383
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:514)
[junit4:junit4] 2> - locked org.apache.hadoop.ipc.Server$Listener$Reader@6cf5d77f
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:500)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer for 'NameNode' metrics system" ID=201 TIMED_WAITING on java.util.TaskQueue@69b03695
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@69b03695
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "SUITE-BasicHdfsTest-seed#[D7E89E61448FA00D]-worker" ID=196 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdown(DataNode.java:1130)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdownDataNodes(MiniDFSCluster.java:1437)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1417)
[junit4:junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:65)
[junit4:junit4] 2> at org.apache.solr.cloud.hdfs.BasicHdfsTest.teardownClass(BasicHdfsTest.java:58)
[junit4:junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] 2> at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "SUITE-BasicHdfsTest-seed#[D7E89E61448FA00D]" ID=195 RUNNABLE
[junit4:junit4] 2> at sun.management.ThreadImpl.dumpThreads0(Native Method)
[junit4:junit4] 2> at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:655)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$800(ThreadLeakControl.java:60)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:396)
[junit4:junit4] 2> - locked java.lang.Object@45fd2247
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:556)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:79)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:492)
[junit4:junit4] 2>
[junit4:junit4] 2> "Signal Dispatcher" ID=5 RUNNABLE
[junit4:junit4] 2>
[junit4:junit4] 2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@2fcfa668
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.ReferenceQueue$Lock@2fcfa668
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
[junit4:junit4] 2> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
[junit4:junit4] 2>
[junit4:junit4] 2> "Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@69b60e5d
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.Reference$Lock@69b60e5d
[junit4:junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4:junit4] 2> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
[junit4:junit4] 2>
[junit4:junit4] 2> "main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@446d9b56
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@446d9b56
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1258)
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1332)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:502)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:399)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:180)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:275)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4:junit4] 2>
[junit4:junit4] 2> ^^==============================================
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=DefaultSimilarity, locale=ar, timezone=America/Indianapolis
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=50,free=72752336,total=239075328
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ChaosMonkeyNothingIsSafeTest, ResourceLoaderTest, FieldMutatingUpdateProcessorTest, TestQueryUtils, DistributedSpellCheckComponentTest, TestSolrQueryParserDefaultOperatorResource, TestFieldResource, JsonLoaderTest, SpatialFilterTest, IndexSchemaTest, TestLFUCache, BasicHdfsTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicHdfsTest -Dtests.seed=D7E89E61448FA00D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar -Dtests.timezone=America/Indianapolis -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J0 | BasicHdfsTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
[junit4:junit4] Completed on J0 in 7200.12s, 1 test, 1 error <<< FAILURES!
[...truncated 68 lines...]
[junit4:junit4] Suite: org.apache.solr.TestSolrCoreProperties
[junit4:junit4] 2> 7399951 T636 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 7399954 T636 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35713
[junit4:junit4] 2> 7399955 T636 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 7399955 T636 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 7399955 T636 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.TestSolrCoreProperties-1372357202682
[junit4:junit4] 2> 7399956 T636 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.TestSolrCoreProperties-1372357202682/solr.xml
[junit4:junit4] 2> 7399956 T636 oasc.CoreContainer.<init> New CoreContainer 1836996732
[junit4:junit4] 2> 7399957 T636 oasc.CoreContainer$Initializer.initialize no solr.xml found. using default old-style solr.xml
[junit4:junit4] 2> 7399957 T636 oasc.CoreContainer.load Loading CoreContainer using Solr Home: './org.apache.solr.TestSolrCoreProperties-1372357202682/'
[junit4:junit4] 2> 7399957 T636 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.TestSolrCoreProperties-1372357202682/'
[junit4:junit4] 2> 7400044 T636 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 7400045 T636 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 7400045 T636 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 7400045 T636 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 7400046 T636 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 7400046 T636 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 7400046 T636 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 7400047 T636 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 7400047 T636 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 7400047 T636 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 7400060 T636 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 7400064 T646 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1
[junit4:junit4] 2> 7400065 T646 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/'
[junit4:junit4] 2> 7400118 T646 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 7400156 T646 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 7400157 T646 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 7400160 T646 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 7400176 T646 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 7400177 T646 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 7400177 T646 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/, dataDir=./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/
[junit4:junit4] 2> 7400177 T646 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
[junit4:junit4] 2> 7400178 T646 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data
[junit4:junit4] 2> 7400178 T646 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index/
[junit4:junit4] 2> 7400178 T646 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 7400179 T646 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index
[junit4:junit4] 2> 7400180 T646 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@792c1f0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2dfc93ef),segFN=segments_1,generation=1}
[junit4:junit4] 2> 7400180 T646 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 7400181 T646 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 7400181 T646 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4] 2> 7400181 T646 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4] 2> 7400181 T646 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4] 2> 7400182 T646 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 7400192 T646 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 7400195 T646 oass.SolrIndexSearcher.<init> Opening Searcher@20945019 main
[junit4:junit4] 2> 7400195 T646 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 7400195 T646 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 7400196 T646 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7400196 T647 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20945019 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 7400196 T636 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 7400197 T636 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 7400197 T636 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C264 name=collection1 org.apache.solr.core.SolrCore@6996d0ff
[junit4:junit4] 2> 7400216 T639 C264 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4:junit4] 2> 7400217 T636 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1836996732
[junit4:junit4] 2> 7400218 T636 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6996d0ff
[junit4:junit4] 2> 7400218 T636 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 7400219 T636 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 7400219 T636 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 7400219 T636 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 7400220 T636 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 7400220 T636 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 7400221 T636 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data [CachedDir<<refCount=0;path=./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data;done=false>>]
[junit4:junit4] 2> 7400221 T636 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data
[junit4:junit4] 2> 7400221 T636 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index [CachedDir<<refCount=0;path=./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index;done=false>>]
[junit4:junit4] 2> 7400222 T636 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.TestSolrCoreProperties-1372357202682/collection1/data/index
[junit4:junit4] 2> 7400222 T636 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCoreProperties -Dtests.method=testSimple -Dtests.seed=D7E89E61448FA00D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=da_DK -Dtests.timezone=Indian/Reunion -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 0.36s J0 | TestSolrCoreProperties.testSimple <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: System properties invariant violated.
[junit4:junit4] > Missing keys:
[junit4:junit4] > tests.shardhandler.randomSeed=6269827861127365263
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D:EF5BBA9F637C74DC]:0)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=da_DK, timezone=Indian/Reunion
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=1,free=196745944,total=385875968
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ChaosMonkeyNothingIsSafeTest, ResourceLoaderTest, FieldMutatingUpdateProcessorTest, TestQueryUtils, DistributedSpellCheckComponentTest, TestSolrQueryParserDefaultOperatorResource, TestFieldResource, JsonLoaderTest, SpatialFilterTest, IndexSchemaTest, TestLFUCache, BasicHdfsTest, TestStressVersions, TestJoin, FastVectorHighlighterTest, PrimitiveFieldTypeTest, BadComponentTest, TestSolrIndexConfig, MBeansHandlerTest, SolrCoreTest, SolrInfoMBeanTest, MinimalSchemaTest, PrimUtilsTest, RegexBoostProcessorTest, WordBreakSolrSpellCheckerTest, SOLR749Test, TestPHPSerializedResponseWriter, TestSerializedLuceneMatchVersion, LukeRequestHandlerTest, FileBasedSpellCheckerTest, LoggingHandlerTest, CSVRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestSolrCoreProperties]
[junit4:junit4] Completed on J0 in 0.38s, 1 test, 1 failure <<< FAILURES!
[...truncated 21 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4:junit4] 2> 228861 T795 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /si/qq
[junit4:junit4] 2> 230480 T795 oahu.NativeCodeLoader.<clinit> WARN Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4:junit4] 2> 230973 T795 oahc.Configuration.warnOnceIfDeprecated WARN hadoop.configured.node.mapping is deprecated. Instead, use net.topology.configured.node.mapping
[junit4:junit4] 2> 232166 T795 oahmi.MetricsConfig.loadFirst WARN Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4:junit4] 2> 234765 T795 oml.Slf4jLog.info Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4:junit4] 2> 235046 T795 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 235146 T795 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/hdfs to ./Jetty_localhost_13680_hdfs____39negm/webapp
[junit4:junit4] 2> 235303 T795 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 235496 T795 oml.Slf4jLog.info Started SelectChannelConnector@localhost:13680
[junit4:junit4] 2> 236378 T795 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 236437 T795 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/datanode to ./Jetty_localhost_13682_datanode____.6c0v63/webapp
[junit4:junit4] 2> 236524 T795 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 236599 T795 oml.Slf4jLog.info Started SelectChannelConnector@localhost:13682
[junit4:junit4] 2> 237707 T795 oml.Slf4jLog.info jetty-6.1.26
[junit4:junit4] 2> 237837 T795 oml.Slf4jLog.info Extract jar:file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/test-framework/lib/hadoop-hdfs-2.0.5-alpha-tests.jar!/webapps/datanode to ./Jetty_localhost_13685_datanode____.kw2sso/webapp
[junit4:junit4] 2> 237967 T795 oml.Slf4jLog.info NO JSP Support for /, did not find org.apache.jasper.servlet.JspServlet
[junit4:junit4] 2> 238100 T795 oml.Slf4jLog.info Started SelectChannelConnector@localhost:13685
[junit4:junit4] 2> 242538 T822 oahhsb.BlockManager.processReport BLOCK* processReport: from DatanodeRegistration(140.211.11.196, storageID=DS-253281107-140.211.11.196-13681-1372350043552, infoPort=13682, ipcPort=13683, storageInfo=lv=-40;cid=testClusterID;nsid=673193632;c=0), blocks: 0, processing time: 22 msecs
[junit4:junit4] 2> 242544 T823 oahhsb.BlockManager.processReport BLOCK* processReport: from DatanodeRegistration(140.211.11.196, storageID=DS-365130897-140.211.11.196-13684-1372350043525, infoPort=13685, ipcPort=13686, storageInfo=lv=-40;cid=testClusterID;nsid=673193632;c=0), blocks: 0, processing time: 0 msecs
[junit4:junit4] 1> Formatting using clusterid: testClusterID
[junit4:junit4] IGNOR/A 0.01s J1 | HdfsCollectionsAPIDistributedZkTest.testDistribSearch
[junit4:junit4] > Assumption #1: 'nightly' test group is disabled (@Nightly)
[junit4:junit4] 2> 242572 T795 oahhsd.DirectoryScanner.shutdown WARN DirectoryScanner: shutdown has been called
[junit4:junit4] 2> 242631 T795 oml.Slf4jLog.info Stopped SelectChannelConnector@localhost:0
[junit4:junit4] 2> 245472 T865 oahhsd.BPServiceActor.run WARN Ending block pool service for: Block pool BP-980596107-140.211.11.196-1372350034001 (storage id DS-365130897-140.211.11.196-13684-1372350043525) service to lucene.zones.apache.org/140.211.11.196:13679
[junit4:junit4] 2> 7200080 T794 ccr.ThreadLeakControl$2.evaluate WARNING Suite execution timed out: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4:junit4] 2> ==== jstack at approximately timeout time ====
[junit4:junit4] 2> "java.util.concurrent.ThreadPoolExecutor$Worker@669e74a5[State = -1, empty queue]" ID=922 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@38d12fae
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@38d12fae
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "refreshUsed-/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1372350031558/hdfsBaseDir/data/data2/current/BP-980596107-140.211.11.196-1372350034001" ID=902 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:85)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "refreshUsed-/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1372350031558/hdfsBaseDir/data/data1/current/BP-980596107-140.211.11.196-1372350034001" ID=899 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.fs.DU$DURefreshThread.run(DU.java:85)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "java.util.concurrent.ThreadPoolExecutor$Worker@138550b8[State = -1, empty queue]" ID=895 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2d9165a6
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2d9165a6
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2082)
[junit4:junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1090)
[junit4:junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Thread-386" ID=888 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:78)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Parameter Sending Thread #0" ID=877 TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@24a88ca9
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.concurrent.SynchronousQueue$TransferStack@24a88ca9
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
[junit4:junit4] 2> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Client (396509229) connection to lucene.zones.apache.org/140.211.11.196:13679 from hudson" ID=876 TIMED_WAITING on org.apache.hadoop.ipc.Client$Connection@35aabe0c
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.apache.hadoop.ipc.Client$Connection@35aabe0c
[junit4:junit4] 2> at org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:799)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:843)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@79c8692b" ID=858 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4] 2> - locked java.lang.Object@5ca3215a
[junit4:junit4] 2> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:99)
[junit4:junit4] 2> - locked java.lang.Object@59e2cb39
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:136)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 9 on 13683" ID=851 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 8 on 13683" ID=850 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 7 on 13683" ID=849 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 6 on 13683" ID=848 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 5 on 13683" ID=847 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 4 on 13683" ID=846 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 3 on 13683" ID=845 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 2 on 13683" ID=844 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 1 on 13683" ID=843 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 0 on 13683" ID=842 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@73b9a762
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server listener on 13683" ID=838 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@7e05049f
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@7b72a35b
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@7e3d5797
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener.run(Server.java:627)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server Responder" ID=840 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@766c9ec9
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@73d4cd33
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@9b131c7
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:811)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.run(Server.java:794)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.datanode.DataXceiverServer@5b61e960" ID=834 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:226)
[junit4:junit4] 2> - locked java.lang.Object@503b04c9
[junit4:junit4] 2> at sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:99)
[junit4:junit4] 2> - locked java.lang.Object@52b75145
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:136)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "DataNode: [file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1372350031558/hdfsBaseDir/data/data1,file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_1372350031558/hdfsBaseDir/data/data2] heartbeating to lucene.zones.apache.org/140.211.11.196:13679" ID=841 TIMED_WAITING on java.util.HashMap@138b370
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.HashMap@138b370
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:573)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:676)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Socket Reader #1 for port 13683" ID=839 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@7ed16e39
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@1a761bfa
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@31fe077f
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:514)
[junit4:junit4] 2> - locked org.apache.hadoop.ipc.Server$Listener$Reader@1c9e3c7d
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:500)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer-1" ID=837 TIMED_WAITING on java.util.TaskQueue@104dff84
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@104dff84
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "74227228@qtp-1481744978-1 - Acceptor0 SelectChannelConnector@localhost:13682" ID=836 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@2eecf0f7
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@404cf664
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@3853d938
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
[junit4:junit4] 2> at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
[junit4:junit4] 2> at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[junit4:junit4] 2>
[junit4:junit4] 2> "402963755@qtp-1481744978-0" ID=835 TIMED_WAITING on org.mortbay.thread.QueuedThreadPool$PoolThread@1804bd2b
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.mortbay.thread.QueuedThreadPool$PoolThread@1804bd2b
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor@393fb5cd" ID=826 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem$NameNodeResourceMonitor.run(FSNamesystem.java:3492)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@370cb985" ID=825 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor.run(LeaseManager.java:419)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 9 on 13679" ID=824 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 8 on 13679" ID=823 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 7 on 13679" ID=822 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 6 on 13679" ID=821 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 5 on 13679" ID=820 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 4 on 13679" ID=819 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 3 on 13679" ID=818 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 2 on 13679" ID=817 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 1 on 13679" ID=816 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server handler 0 on 13679" ID=815 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4:junit4] 2> - waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@70c23b78
[junit4:junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1718)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server listener on 13679" ID=803 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@225eeefe
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@69c7753
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@1c95c2b0
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener.run(Server.java:627)
[junit4:junit4] 2>
[junit4:junit4] 2> "IPC Server Responder" ID=805 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@19fab791
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@493e7b94
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@332c43f7
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.doRunLoop(Server.java:811)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Responder.run(Server.java:794)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer-0" ID=814 TIMED_WAITING on java.util.TaskQueue@5fce7d1d
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@5fce7d1d
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "1511953677@qtp-1535058989-1 - Acceptor0 SelectChannelConnector@localhost:13680" ID=813 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@7b471f97
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@138e2ac7
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@5373db60
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
[junit4:junit4] 2> at org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
[junit4:junit4] 2> at org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
[junit4:junit4] 2> at org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
[junit4:junit4] 2>
[junit4:junit4] 2> "1632405826@qtp-1535058989-0" ID=812 TIMED_WAITING on org.mortbay.thread.QueuedThreadPool$PoolThread@614c8942
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on org.mortbay.thread.QueuedThreadPool$PoolThread@614c8942
[junit4:junit4] 2> at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:626)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@49d648ca" ID=801 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3079)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor@1344eba5" ID=802 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager$Monitor.run(HeartbeatManager.java:295)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor@2abf22b" ID=810 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.DecommissionManager$Monitor.run(DecommissionManager.java:76)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor@41cd0d5f" ID=809 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.PendingReplicationBlocks$PendingReplicationMonitor.run(PendingReplicationBlocks.java:211)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Socket Reader #1 for port 13679" ID=804 RUNNABLE (in native code)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4] 2> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4] 2> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4] 2> - locked sun.nio.ch.Util$2@3cff6cec
[junit4:junit4] 2> - locked java.util.Collections$UnmodifiableSet@2548193d
[junit4:junit4] 2> - locked sun.nio.ch.KQueueSelectorImpl@5c7cfd49
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:102)
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:514)
[junit4:junit4] 2> - locked org.apache.hadoop.ipc.Server$Listener$Reader@11d8f30f
[junit4:junit4] 2> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:500)
[junit4:junit4] 2>
[junit4:junit4] 2> "Timer for 'NameNode' metrics system" ID=800 TIMED_WAITING on java.util.TaskQueue@6c94030
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - timed waiting on java.util.TaskQueue@6c94030
[junit4:junit4] 2> at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4] 2> at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4] 2>
[junit4:junit4] 2> "SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D7E89E61448FA00D]-worker" ID=795 TIMED_WAITING
[junit4:junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdown(DataNode.java:1130)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdownDataNodes(MiniDFSCluster.java:1437)
[junit4:junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1417)
[junit4:junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:65)
[junit4:junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4:junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] 2> at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:700)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[D7E89E61448FA00D]" ID=794 RUNNABLE
[junit4:junit4] 2> at sun.management.ThreadImpl.dumpThreads0(Native Method)
[junit4:junit4] 2> at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:655)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$800(ThreadLeakControl.java:60)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:396)
[junit4:junit4] 2> - locked java.lang.Object@3a3d66f0
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:556)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:79)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$1.run(RandomizedRunner.java:492)
[junit4:junit4] 2>
[junit4:junit4] 2> "Signal Dispatcher" ID=5 RUNNABLE
[junit4:junit4] 2>
[junit4:junit4] 2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@69512cb
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.ReferenceQueue$Lock@69512cb
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
[junit4:junit4] 2> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
[junit4:junit4] 2>
[junit4:junit4] 2> "Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@2504d0cd
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.Reference$Lock@2504d0cd
[junit4:junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4:junit4] 2> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
[junit4:junit4] 2>
[junit4:junit4] 2> "main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@48cf06c0
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@48cf06c0
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1258)
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1332)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:502)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:399)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:180)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:275)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4:junit4] 2>
[junit4:junit4] 2> ^^==============================================
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=zh_SG, timezone=Europe/Amsterdam
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=51,free=227917864,total=400556032
[junit4:junit4] 2> NOTE: All tests run in this JVM: [CollectionsAPIDistributedZkTest, CachingDirectoryFactoryTest, DOMUtilTest, SortByFunctionTest, StatsComponentTest, NotRequiredUniqueKeyTest, BadIndexSchemaTest, TestSolrXMLSerializer, TestRTGBase, TestCloudManagedSchemaAddField, TestSolr4Spatial, HdfsCollectionsAPIDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.seed=D7E89E61448FA00D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh_SG -Dtests.timezone=Europe/Amsterdam -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J1 | HdfsCollectionsAPIDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
[junit4:junit4] Completed on J1 in 7200.14s, 1 test, 1 error, 1 skipped <<< FAILURES!
[...truncated 227 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ClusterStateUpdateTest
[junit4:junit4] 2> 8704957 T4436 oas.SolrTestCaseJ4.setUp ###Starting testCoreRegistration
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699
[junit4:junit4] 2> 8704958 T4436 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 8704959 T4437 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 8705060 T4436 oasc.ZkTestServer.run start zk server on port:58570
[junit4:junit4] 2> 8705061 T4436 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8705100 T4443 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61803ffd name:ZooKeeperConnection Watcher:127.0.0.1:58570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8705100 T4436 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8705101 T4436 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 8705119 T4436 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8705120 T4445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2890e2a9 name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8705121 T4436 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8705121 T4436 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 8705132 T4436 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 8705135 T4436 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 8705137 T4436 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 8705146 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 8705147 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 8705153 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 8705154 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 8705258 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 8705258 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 8705261 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 8705262 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 8705273 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 8705274 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 8705276 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 8705277 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 8705280 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 8705281 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 8705284 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 8705284 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 8705287 T4436 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 8705288 T4436 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 8705299 T4436 oasc.ClusterStateUpdateTest.setUp ####SETUP_START testCoreRegistration
[junit4:junit4] 2> 8705303 T4436 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8705304 T4436 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr
[junit4:junit4] 2> 8705304 T4436 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/solr.xml
[junit4:junit4] 2> 8705304 T4436 oasc.CoreContainer.<init> New CoreContainer 109163318
[junit4:junit4] 2> 8705305 T4436 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4] 2> 8705305 T4436 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/'
[junit4:junit4] 2> 8705434 T4436 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8705434 T4436 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8705435 T4436 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8705435 T4436 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8705436 T4436 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8705436 T4436 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8705436 T4436 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8705437 T4436 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8705437 T4436 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8705438 T4436 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8705455 T4436 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 8705456 T4436 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58570/solr
[junit4:junit4] 2> 8705456 T4436 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8705457 T4436 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8705484 T4447 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79f4a60c name:ZooKeeperConnection Watcher:127.0.0.1:58570 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8705485 T4436 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8705486 T4436 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8705498 T4436 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8705500 T4449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66dbd989 name:ZooKeeperConnection Watcher:127.0.0.1:58570/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8705501 T4436 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8705503 T4436 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 8705513 T4436 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 8705516 T4436 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 8705519 T4436 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:1661_waxvc%2Fl
[junit4:junit4] 2> 8705529 T4436 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:1661_waxvc%2Fl
[junit4:junit4] 2> 8705531 T4436 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 8705543 T4436 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 8705551 T4436 oasc.Overseer.start Overseer (id=89938887161020419-127.0.0.1:1661_waxvc%2Fl-n_0000000000) starting
[junit4:junit4] 2> 8705555 T4436 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 8705558 T4451 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 8705559 T4436 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 8705561 T4436 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 8705563 T4436 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 8705565 T4450 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 8705569 T4452 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 8705570 T4452 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8705571 T4452 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 8707068 T4450 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8707069 T4450 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:1661_waxvc%2Fl_collection1",
[junit4:junit4] 2> "numShards":null,
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:1661_waxvc%2Fl",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:1661/waxvc/l"}
[junit4:junit4] 2> 8707069 T4450 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 8707085 T4449 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 8707573 T4452 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1
[junit4:junit4] 2> 8707573 T4452 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 8707574 T4452 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 8707574 T4452 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 8707575 T4452 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/'
[junit4:junit4] 2> 8707577 T4452 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
[junit4:junit4] 2> 8707577 T4452 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 8707634 T4452 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 8707709 T4452 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 8707810 T4452 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 8707816 T4452 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 8708463 T4452 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 8708475 T4452 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 8708480 T4452 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 8708493 T4452 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 8708497 T4452 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 8708502 T4452 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 8708503 T4452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 8708503 T4452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 8708503 T4452 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 8708504 T4452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 8708504 T4452 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 8708505 T4452 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 8708505 T4452 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699/data1/
[junit4:junit4] 2> 8708505 T4452 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6134cd50
[junit4:junit4] 2> 8708506 T4452 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699/data1
[junit4:junit4] 2> 8708507 T4452 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699/data1/index/
[junit4:junit4] 2> 8708507 T4452 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699/data1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 8708507 T4452 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-ClusterStateUpdateTest-1372358507699/data1/index
[junit4:junit4] 2> 8708510 T4452 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirWrapper(org.apache.lucene.store.RAMDirectory@4ddf47f6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@724e7bdf)),segFN=segments_1,generation=1}
[junit4:junit4] 2> 8708511 T4452 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 8708512 T4452 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4] 2> 8708514 T4452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 8708514 T4452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4] 2> 8708514 T4452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 8708515 T4452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4] 2> 8708516 T4452 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 8708516 T4452 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 8708516 T4452 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 8708517 T4452 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 8708517 T4452 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 8708518 T4452 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4] 2> 8708518 T4452 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4] 2> 8708519 T4452 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 8708519 T4452 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4] 2> 8708520 T4452 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4] 2> 8708520 T4452 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4] 2> 8708521 T4452 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 8708521 T4452 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708522 T4452 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708522 T4452 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708523 T4452 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708523 T4452 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708524 T4452 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708524 T4452 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708525 T4452 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4] 2> 8708525 T4452 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4] 2> 8708526 T4452 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4] 2> 8708542 T4452 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 8708548 T4452 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4] 2> 8708549 T4452 oasc.SolrCore.initDeprecatedSupport WARN adding ShowFileRequestHandler with hidden files: [OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4] 2> 8708552 T4452 oass.SolrIndexSearcher.<init> Opening Searcher@678c3ffc main
[junit4:junit4] 2> 8708553 T4452 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 8708553 T4452 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 8708554 T4452 oashc.SpellCheckComponent.inform Initializing spell checkers
[juni
[...truncated too long message...]
hadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> "Signal Dispatcher" ID=5 RUNNABLE
[junit4:junit4] 2>
[junit4:junit4] 2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@2fcfa668
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.ReferenceQueue$Lock@2fcfa668
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135)
[junit4:junit4] 2> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151)
[junit4:junit4] 2> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189)
[junit4:junit4] 2>
[junit4:junit4] 2> "Reference Handler" ID=2 WAITING on java.lang.ref.Reference$Lock@69b60e5d
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on java.lang.ref.Reference$Lock@69b60e5d
[junit4:junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4:junit4] 2> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133)
[junit4:junit4] 2>
[junit4:junit4] 2> "main" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$1@22288a5
[junit4:junit4] 2> at java.lang.Object.wait(Native Method)
[junit4:junit4] 2> - waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$1@22288a5
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1258)
[junit4:junit4] 2> at java.lang.Thread.join(Thread.java:1332)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:502)
[junit4:junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:399)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:180)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:275)
[junit4:junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
[junit4:junit4] 2>
[junit4:junit4] 2> ^^==============================================
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=es_US, timezone=Asia/Qyzylorda
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=48,free=269154632,total=513802240
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ChaosMonkeyNothingIsSafeTest, ResourceLoaderTest, FieldMutatingUpdateProcessorTest, TestQueryUtils, DistributedSpellCheckComponentTest, TestSolrQueryParserDefaultOperatorResource, TestFieldResource, JsonLoaderTest, SpatialFilterTest, IndexSchemaTest, TestLFUCache, BasicHdfsTest, TestStressVersions, TestJoin, FastVectorHighlighterTest, PrimitiveFieldTypeTest, BadComponentTest, TestSolrIndexConfig, MBeansHandlerTest, SolrCoreTest, SolrInfoMBeanTest, MinimalSchemaTest, PrimUtilsTest, RegexBoostProcessorTest, WordBreakSolrSpellCheckerTest, SOLR749Test, TestPHPSerializedResponseWriter, TestSerializedLuceneMatchVersion, LukeRequestHandlerTest, FileBasedSpellCheckerTest, LoggingHandlerTest, CSVRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestSolrCoreProperties, TestFunctionQuery, BadCopyFieldTest, CoreAdminHandlerTest, DocValuesTest, ScriptEngineTest, BinaryUpdateRequestHandlerTest, OverseerTest, PluginInfoTest, TestXIncludeConfig, TestDocumentBuilder, ExternalFileFieldSortTest, TestPhraseSuggestions, TestReplicationHandler, TestPseudoReturnFields, TestSort, TestValueSourceCache, SystemInfoHandlerTest, TestSearchPerf, TestLRUCache, TestSolrXmlPersistence, TestWriterPerf, SynonymTokenizerTest, TestAddFieldRealTimeGet, SolrIndexSplitterTest, QueryParsingTest, SoftAutoCommitTest, TestReversedWildcardFilterFactory, ShardRoutingTest, PreAnalyzedUpdateProcessorTest, AssignTest, DistributedTermsComponentTest, TestGroupingSearch, TestSweetSpotSimilarityFactory, DateFieldTest, TestTrie, TestSchemaVersionResource, DistributedQueryElevationComponentTest, AutoCommitTest, BasicFunctionalityTest, DirectSolrConnectionTest, OverseerCollectionProcessorTest, ZkControllerTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, ZkCLITest, SliceStateUpdateTest, TestDistributedSearch, TestFaceting, TestRecovery, TestHashPartitioner, TestRealTimeGet, TestReload, HardAutoCommitTest, TestCoreContainer, SimpleFacetsTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, TestFiltering, TestLazyCores, TestBadConfig, HighlighterTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, CurrencyFieldXmlFileTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTest, TestStressLucene, SuggesterWFSTTest, PolyFieldTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, CacheHeaderTest, TestSurroundQueryParser, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TermsComponentTest, RequestHandlersTest, DocumentBuilderTest, TermVectorComponentTest, TestIndexingPerformance, MoreLikeThisHandlerTest, TestSolrQueryParser, TestArbitraryIndexDir, TestJmxIntegration, TestCollationField, ReturnFieldsTest, JSONWriterTest, TestCSVResponseWriter, TestPartialUpdateDeduplication, TestComponentsName, SearchHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, TestQuerySenderListener, TestPropInjectDefaults, AlternateDirectoryTest, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestStressRecovery, TestSolrDeletionPolicy2, TestDocSet, NumericFieldsTest, TestConfig, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestPluginEnable, TimeZoneUtilsTest, TestFastOutputStream, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, SpellingQueryConverterTest, RAMDirectoryFactoryTest, ClusterStateTest, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, CircularListTest, AliasIntegrationTest, HdfsBasicDistributedZk2Test]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsBasicDistributedZk2Test -Dtests.seed=D7E89E61448FA00D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=Asia/Qyzylorda -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J0 | HdfsBasicDistributedZk2Test (suite) <<<
[junit4:junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D7E89E61448FA00D]:0)
[junit4:junit4] Completed on J0 in 7200.15s, 1 test, 1 error, 1 skipped <<< FAILURES!
[...truncated 16 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1243: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:887: There were test failures: 312 suites, 1315 tests, 6 suite-level errors, 2 failures, 193 ignored (9 assumptions)
Total time: 297 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4102 - Still Failing
Posted by Mark Miller <ma...@gmail.com>.
FreeBSD issue it looks - that just started recently - an http connect attempt that won't time out even though there is a timeout set. I need to try and dig into it on my FreeBSD machine when I get a chance.
- Mark
On Jul 1, 2013, at 9:16 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4102/
>
> 2 tests failed.
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
>
>
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> There are still zombie threads that couldn't be terminated: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
>
>
>
>
> Build Log:
> [...truncated 9599 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4] 2> 732978 T2067 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zm/x
> [junit4:junit4] 2> 732981 T2067 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372684133825
> [junit4:junit4] 2> 732983 T2067 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4] 2> 732983 T2068 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4] 2> 733084 T2067 oasc.ZkTestServer.run start zk server on port:17602
> [junit4:junit4] 2> 733085 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733090 T2074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3329a45 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733090 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733091 T2067 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4] 2> 733108 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733110 T2076 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4792a0c9 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733110 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733111 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4] 2> 733115 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4] 2> 733118 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4] 2> 733121 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4] 2> 733125 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 733125 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 733131 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4] 2> 733132 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4] 2> 733240 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 733240 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 733245 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4] 2> 733246 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4] 2> 733249 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4] 2> 733250 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4] 2> 733257 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 733258 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 733261 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 733262 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 733265 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 733265 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 733273 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 733273 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 733458 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 733461 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19896
> [junit4:junit4] 2> 733462 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/solr.xml
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.<init> New CoreContainer 1845363899
> [junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
> [junit4:junit4] 2> 733464 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
> [junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 733553 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 733553 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 733564 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 733565 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 733565 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 733566 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733569 T2087 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5766e024 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733569 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733584 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 733597 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 733599 T2089 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6974bbb6 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 733599 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 733602 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4:junit4] 2> 733611 T2067 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4:junit4] 2> 733614 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4] 2> 733621 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19896_zm%2Fx
> [junit4:junit4] 2> 733623 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19896_zm%2Fx
> [junit4:junit4] 2> 733627 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4] 2> 733642 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 733646 T2067 oasc.Overseer.start Overseer (id=89960227394879491-127.0.0.1:19896_zm%2Fx-n_0000000000) starting
> [junit4:junit4] 2> 733656 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4:junit4] 2> 733667 T2091 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 733667 T2067 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4] 2> 733670 T2067 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4:junit4] 2> 733677 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 733680 T2090 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 733685 T2092 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 735184 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 735184 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
> [junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 735201 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 735688 T2092 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 735688 T2092 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1
> [junit4:junit4] 2> 735689 T2092 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4] 2> 735690 T2092 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 735690 T2092 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4] 2> 735692 T2092 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/'
> [junit4:junit4] 2> 735693 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/README' to classloader
> [junit4:junit4] 2> 735694 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 735746 T2092 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 735808 T2092 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 735911 T2092 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 735918 T2092 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 736551 T2092 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 736564 T2092 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 736568 T2092 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 736585 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 736590 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 736595 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 736597 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 736598 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 736598 T2092 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 736598 T2092 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/
> [junit4:junit4] 2> 736599 T2092 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 736600 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data
> [junit4:junit4] 2> 736600 T2092 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index/
> [junit4:junit4] 2> 736600 T2092 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 736601 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index
> [junit4:junit4] 2> 736605 T2092 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 736606 T2092 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 736608 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 736628 T2092 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 736636 T2092 oass.SolrIndexSearcher.<init> Opening Searcher@782f5cc2 main
> [junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 736641 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782f5cc2 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 736642 T2092 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 736643 T2092 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19896/zm/x collection:control_collection shard:shard1
> [junit4:junit4] 2> 736643 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4] 2> 736662 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 736665 T2092 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19896/zm/x/collection1/
> [junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncToMe http://127.0.0.1:19896/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 736666 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19896/zm/x/collection1/ shard1
> [junit4:junit4] 2> 736667 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4] 2> 736707 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 736715 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.register We are http://127.0.0.1:19896/zm/x/collection1/ and leader is http://127.0.0.1:19896/zm/x/collection1/
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19896/zm/x
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 736724 T2092 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 736725 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 736726 T2092 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 736729 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 736746 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4] 2> 736748 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 736750 T2096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78c4fabc name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 736750 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 736752 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 736755 T2067 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4] 2> 737034 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 737037 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16933
> [junit4:junit4] 2> 737038 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598
> [junit4:junit4] 2> 737040 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/solr.xml
> [junit4:junit4] 2> 737040 T2067 oasc.CoreContainer.<init> New CoreContainer 1071384450
> [junit4:junit4] 2> 737041 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
> [junit4:junit4] 2> 737041 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
> [junit4:junit4] 2> 737171 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 737176 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 737193 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 737194 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 737194 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 737195 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 737198 T2107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fa8922 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 737199 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 737214 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 737227 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 737229 T2109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66491c84 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 737229 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 737241 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 738222 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 738223 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
> [junit4:junit4] 2> 738245 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738245 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738245 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738247 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16933_zm%2Fx
> [junit4:junit4] 2> 738256 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16933_zm%2Fx
> [junit4:junit4] 2> 738260 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738260 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 738260 T2096 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738261 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 738262 T2109 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738262 T2089 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 738273 T2110 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 739760 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 739781 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 739781 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 739781 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 740276 T2110 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 740276 T2110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1
> [junit4:junit4] 2> 740276 T2110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 740277 T2110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 740277 T2110 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 740279 T2110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/'
> [junit4:junit4] 2> 740280 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/README' to classloader
> [junit4:junit4] 2> 740281 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 740333 T2110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 740395 T2110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 740497 T2110 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 740504 T2110 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 741168 T2110 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 741181 T2110 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 741185 T2110 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 741202 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 741207 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 741211 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 741214 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 741215 T2110 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 741216 T2110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/
> [junit4:junit4] 2> 741216 T2110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 741217 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1
> [junit4:junit4] 2> 741217 T2110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index/
> [junit4:junit4] 2> 741217 T2110 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 741218 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index
> [junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 741225 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 741243 T2110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 741250 T2110 oass.SolrIndexSearcher.<init> Opening Searcher@68a89d83 main
> [junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 741255 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a89d83 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 741257 T2110 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 741257 T2110 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16933/zm/x collection:collection1 shard:shard1
> [junit4:junit4] 2> 741258 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4] 2> 741278 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 741281 T2110 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncToMe http://127.0.0.1:16933/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 741282 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16933/zm/x/collection1/ shard1
> [junit4:junit4] 2> 741282 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 741286 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 741295 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741295 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741295 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.register We are http://127.0.0.1:16933/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16933/zm/x
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 741341 T2110 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 741342 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 741344 T2110 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 741345 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 741346 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 741346 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 741609 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 741612 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45733
> [junit4:junit4] 2> 741613 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 741613 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 741614 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204
> [junit4:junit4] 2> 741614 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/solr.xml
> [junit4:junit4] 2> 741614 T2067 oasc.CoreContainer.<init> New CoreContainer 2090646828
> [junit4:junit4] 2> 741615 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
> [junit4:junit4] 2> 741615 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
> [junit4:junit4] 2> 741736 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 741737 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 741741 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 741755 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 741756 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 741756 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 741758 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 741761 T2123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd83276 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 741761 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 741779 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 741789 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 741792 T2125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c940331 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 741792 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 741798 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 742801 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45733_zm%2Fx
> [junit4:junit4] 2> 742802 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 742821 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45733_zm%2Fx
> [junit4:junit4] 2> 742821 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
> [junit4:junit4] 2> 742863 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 742863 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 742863 T2125 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742863 T2096 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742864 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742882 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742882 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742922 T2109 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742923 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742926 T2089 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 742927 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 742928 T2126 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 742929 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 742957 T2126 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 744428 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 744450 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744450 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 744959 T2126 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 744959 T2126 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1
> [junit4:junit4] 2> 744959 T2126 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 744960 T2126 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 744960 T2126 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 744962 T2126 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/'
> [junit4:junit4] 2> 744963 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/README' to classloader
> [junit4:junit4] 2> 744964 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 745016 T2126 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 745078 T2126 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 745181 T2126 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 745188 T2126 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 745851 T2126 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 745863 T2126 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 745867 T2126 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 745883 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 745888 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 745892 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 745896 T2126 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 745896 T2126 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/
> [junit4:junit4] 2> 745897 T2126 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 745898 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2
> [junit4:junit4] 2> 745899 T2126 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index/
> [junit4:junit4] 2> 745899 T2126 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 745899 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index
> [junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 745907 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 745907 T2126 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 745924 T2126 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 745931 T2126 oass.SolrIndexSearcher.<init> Opening Searcher@3d1a7e9d main
> [junit4:junit4] 2> 745932 T2126 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 745933 T2126 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 745937 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d1a7e9d main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 745939 T2126 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 745939 T2126 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45733/zm/x collection:collection1 shard:shard2
> [junit4:junit4] 2> 745940 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4] 2> 745958 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncToMe http://127.0.0.1:45733/zm/x/collection1/ has no replicas
> [junit4:junit4] 2> 745963 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45733/zm/x/collection1/ shard2
> [junit4:junit4] 2> 745963 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 747462 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 747480 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747480 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.register We are http://127.0.0.1:45733/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45733/zm/x
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 747506 T2126 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 747507 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 747508 T2126 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 747511 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 747798 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 747801 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45777
> [junit4:junit4] 2> 747802 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 747802 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 747803 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371
> [junit4:junit4] 2> 747803 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/solr.xml
> [junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.<init> New CoreContainer 805132591
> [junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
> [junit4:junit4] 2> 747805 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
> [junit4:junit4] 2> 747936 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 747941 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 747958 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 747959 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 747960 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 747961 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 747993 T2139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616dff74 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 747993 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 748027 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 748037 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 748040 T2141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46e38d21 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 748040 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 748078 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 748987 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 748988 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
> [junit4:junit4] 2> 749008 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749008 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749082 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45777_zm%2Fx
> [junit4:junit4] 2> 749084 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45777_zm%2Fx
> [junit4:junit4] 2> 749087 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749087 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749087 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 749088 T2125 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749089 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 749089 T2109 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749090 T2089 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749096 T2141 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 749104 T2142 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 750514 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 750533 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 750533 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 751106 T2142 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 751106 T2142 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1
> [junit4:junit4] 2> 751106 T2142 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 751107 T2142 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 751107 T2142 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 751109 T2142 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/'
> [junit4:junit4] 2> 751110 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 751111 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/README' to classloader
> [junit4:junit4] 2> 751163 T2142 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 751225 T2142 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 751328 T2142 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 751335 T2142 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 751970 T2142 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 751982 T2142 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 751987 T2142 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 752003 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 752008 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 752013 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 752014 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 752017 T2142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 752017 T2142 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/
> [junit4:junit4] 2> 752017 T2142 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 752018 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3
> [junit4:junit4] 2> 752019 T2142 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index/
> [junit4:junit4] 2> 752019 T2142 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 752020 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index
> [junit4:junit4] 2> 752023 T2142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 752024 T2142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 752026 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 752028 T2142 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 752047 T2142 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 752054 T2142 oass.SolrIndexSearcher.<init> Opening Searcher@70a83d7a main
> [junit4:junit4] 2> 752055 T2142 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 752056 T2142 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 752060 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a83d7a main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 752062 T2142 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 752062 T2142 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45777/zm/x collection:collection1 shard:shard1
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.register We are http://127.0.0.1:45777/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45777/zm/x
> [junit4:junit4] 2> 752066 T2142 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 752067 T2142 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C991 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C991_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 752067 T2144 C991 P45777 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 752068 T2142 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 752069 T2144 C991 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 752069 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 752070 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 752070 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 752071 T2144 C991 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 752096 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 752360 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 752363 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36101
> [junit4:junit4] 2> 752364 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938
> [junit4:junit4] 2> 752366 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/solr.xml
> [junit4:junit4] 2> 752366 T2067 oasc.CoreContainer.<init> New CoreContainer 1659676005
> [junit4:junit4] 2> 752367 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
> [junit4:junit4] 2> 752367 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
> [junit4:junit4] 2> 752496 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 752497 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 752501 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 752501 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 752518 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 752519 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
> [junit4:junit4] 2> 752520 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 752521 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 752524 T2156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cf00b8d name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 752525 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 752539 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 752552 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 752554 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a15d897 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 752555 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 752566 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 753541 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 753542 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 753550 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753550 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753571 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36101_zm%2Fx
> [junit4:junit4] 2> 753573 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36101_zm%2Fx
> [junit4:junit4] 2> 753576 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753576 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 753576 T2125 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753577 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 753578 T2109 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753578 T2089 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753579 T2141 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753579 T2158 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 753587 T2159 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 754099 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
> [junit4:junit4] 2> 754099 T2099 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:45777_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
> [junit4:junit4] 2> 755065 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 755072 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755072 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 755590 T2159 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 755590 T2159 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1
> [junit4:junit4] 2> 755590 T2159 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 755591 T2159 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 755592 T2159 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 755593 T2159 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/'
> [junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/README' to classloader
> [junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 755647 T2159 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 755709 T2159 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 755812 T2159 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 755819 T2159 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> ASYNC NEW_CORE C992 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C992_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 756102 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:16933/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 756102 T2144 C992 P45777 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45777/zm/x START replicas=[http://127.0.0.1:16933/zm/x/collection1/] nUpdates=100
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:16933/zm/x/collection1/. core=collection1
> [junit4:junit4] 2> 756105 T2144 C992 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C993 name=collection1 org.apache.solr.core.SolrCore@215eb2d8 url=http://127.0.0.1:16933/zm/x/collection1 node=127.0.0.1:16933_zm%2Fx C993_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:16933_zm%2Fx, base_url=http://127.0.0.1:16933/zm/x, leader=true}
> [junit4:junit4] 2> 756113 T2100 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 756116 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 756119 T2101 C993 P16933 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 756120 T2101 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 756120 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 756121 T2101 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@4d16c784 realtime
> [junit4:junit4] 2> 756121 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 756122 T2101 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 756122 T2144 C992 P45777 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 756123 T2144 C992 P45777 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 756125 T2102 C993 P16933 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 756125 T2102 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 756128 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 756453 T2159 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 756466 T2159 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 756470 T2159 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 756487 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 756492 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 756496 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 756499 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 756500 T2159 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 756501 T2159 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/
> [junit4:junit4] 2> 756501 T2159 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
> [junit4:junit4] 2> 756502 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4
> [junit4:junit4] 2> 756502 T2159 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index/
> [junit4:junit4] 2> 756503 T2159 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 756503 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index
> [junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 756511 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 756513 T2159 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 756531 T2159 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 756538 T2159 oass.SolrIndexSearcher.<init> Opening Searcher@37878dd4 main
> [junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 756544 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37878dd4 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 756546 T2159 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 756546 T2159 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36101/zm/x collection:collection1 shard:shard2
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.register We are http://127.0.0.1:36101/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36101/zm/x
> [junit4:junit4] 2> 756550 T2159 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 756550 T2159 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C994 name=collection1 org.apache.solr.core.SolrCore@7f9607f7 url=http://127.0.0.1:36101/zm/x/collection1 node=127.0.0.1:36101_zm%2Fx C994_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 756551 T2159 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 756553 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 756554 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 756572 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756573 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 756574 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
> [junit4:junit4] 2> 756575 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756585 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 756585 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
> [junit4:junit4] 2> 756590 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 756601 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756602 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756601 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 756601 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 757575 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
> [junit4:junit4] 2> 757575 T2115 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:36101_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
> [junit4:junit4] 2> 757578 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 758581 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 759578 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:45733/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 759578 T2162 C994 P36101 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36101/zm/x START replicas=[http://127.0.0.1:45733/zm/x/collection1/] nUpdates=100
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 759580 T2162 C994 P36101 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 759580 T2162 C994 P36101 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:45733/zm/x/collection1/. core=collection1
> [junit4:junit4] 2> 759581 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 759584 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> ASYNC NEW_CORE C995 name=collection1 org.apache.solr.core.SolrCore@1e8d62e0 url=http://127.0.0.1:45733/zm/x/collection1 node=127.0.0.1:45733_zm%2Fx C995_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45733_zm%2Fx, base_url=http://127.0.0.1:45733/zm/x, leader=true}
> [junit4:junit4] 2> 759588 T2116 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 759592 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 759594 T2117 C995 P45733 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 759596 T2117 C995 P45733 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 759596 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@496ac93a realtime
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 759597 T2117 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 759598 T2162 C994 P36101 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 759598 T2162 C994 P36101 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 759600 T2118 C995 P45733 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 759600 T2118 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 759602 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 759639 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 759641 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 759642 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
> [junit4:junit4] 2> 759653 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 759653 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 760586 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760587 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
> [junit4:junit4] 2> ASYNC NEW_CORE C996 name=collection1 org.apache.solr.core.SolrCore@565d49dc url=http://127.0.0.1:19896/zm/x/collection1 node=127.0.0.1:19896_zm%2Fx C996_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:19896_zm%2Fx, base_url=http://127.0.0.1:19896/zm/x, leader=true}
> [junit4:junit4] 2> 760599 T2079 C996 P19896 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760600 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760602 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@462edf25 realtime
> [junit4:junit4] 2> 760602 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667266109440)} 0 6
> [junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
> [junit4:junit4] 2> 760626 T2148 C994 P36101 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760627 T2148 C994 P36101 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760629 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@2c01e8e2 realtime
> [junit4:junit4] 2> 760629 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667282886656&update.from=http://127.0.0.1:45733/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 4
> [junit4:junit4] 2> C992_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
> [junit4:junit4] 2> 760630 T2131 C992 P45777 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 760631 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 760632 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6925ec4e realtime
> [junit4:junit4] 2> 760633 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667287080960&update.from=http://127.0.0.1:16933/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 5
> [junit4:junit4] 2> 760634 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 17
> [junit4:junit4] 2> 760634 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 21
> [junit4:junit4] 2> 760636 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760638 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760640 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760642 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760643 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760646 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760648 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760652 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760674 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760675 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760681 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@190c0090 main
> [junit4:junit4] 2> 760681 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760682 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@190c0090 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760682 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667322732544)],commit=} 0 31
> [junit4:junit4] 2> 760689 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667358384128)]} 0 1
> [junit4:junit4] 2> 760690 T2104 C993 P16933 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760692 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760692 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760692 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760693 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760693 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760693 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760695 T2150 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@693e8ddf main
> [junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760696 T2121 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@10432ba8 main
> [junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 760697 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@693e8ddf main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760696 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760698 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10432ba8 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 760697 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760699 T2105 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@6a162992 main
> [junit4:junit4] 2> 760698 T2150 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760700 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760700 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@64c26d35 main
> [junit4:junit4] 2> 760700 T2121 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
> [junit4:junit4] 2> 760701 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760701 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a162992 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760702 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64c26d35 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 760702 T2105 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760703 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760703 T2104 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667358384128)],commit=} 0 18
> [junit4:junit4] 2> 760705 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760707 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760712 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760715 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760717 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760720 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760723 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760724 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760726 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760732 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760732 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760736 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760737 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760743 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760743 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 760746 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760747 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760749 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760755 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760755 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760759 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760760 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760766 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760766 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760769 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760770 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760775 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760778 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760779 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760780 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@54bd13e5 main
> [junit4:junit4] 2> 760781 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760781 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54bd13e5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760782 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667450658816)],commit=} 0 9
> [junit4:junit4] 2> 760795 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 2
> [junit4:junit4] 2> 760796 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 8
> [junit4:junit4] 2> 760796 T2120 C995 P45733 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760798 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760798 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760800 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760800 T2185 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@17b4c540 main
> [junit4:junit4] 2> 760800 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760800 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760802 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17b4c540 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760802 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760801 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760803 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@67e514d main
> [junit4:junit4] 2> 760803 T2133 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760804 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760804 T2185 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760805 T2103 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 760805 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67e514d main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760805 T2133 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760807 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 760806 T2103 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 760808 T2133 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@3cd97ef4 main
> [junit4:junit4] 2> 760809 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760809 T2103 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@1ec58f94 main
> [junit4:junit4] 2> 760810 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd97ef4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760810 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760810 T2133 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
> [junit4:junit4] 2> 760811 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ec58f94 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
> [junit4:junit4] 2> 760811 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
> [junit4:junit4] 2> 760812 T2120 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
> [junit4:junit4] 2> 760813 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760815 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760819 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760823 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760824 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760828 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760832 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760833 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760835 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760841 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760841 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760846 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760847 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 760853 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760853 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760857 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760858 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760859 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760866 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760866 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760870 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760871 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760877 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760878 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760882 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760883 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 760889 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760892 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760893 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760894 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@3b31fd2f main
> [junit4:junit4] 2> 760895 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760895 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b31fd2f main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760896 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667570196480)],commit=} 0 9
> [junit4:junit4] 2> 760907 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 2
> [junit4:junit4] 2> 760908 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45777/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 7
> [junit4:junit4] 2> 760908 T2132 C992 P45777 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 760911 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760911 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760912 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760913 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 760913 T2191 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@5dcf10fa main
> [junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760914 T2151 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@70a07912 main
> [junit4:junit4] 2> 760915 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dcf10fa main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760915 T2187 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 760916 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a07912 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760914 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760916 T2187 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 760915 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760917 T2134 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6b4a3fc7 main
> [junit4:junit4] 2> 760917 T2191 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 760918 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760918 T2187 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@5338fea9 main
> [junit4:junit4] 2> 760918 T2151 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 760919 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760919 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b4a3fc7 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760920 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5338fea9 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
> [junit4:junit4] 2> 760921 T2134 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760921 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 760922 T2132 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
> [junit4:junit4] 2> 760923 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760925 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760929 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760931 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760933 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760936 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760939 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760940 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760942 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760948 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760949 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760952 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760953 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760958 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 760959 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760962 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760963 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760964 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 760970 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760972 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760975 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760976 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 760981 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 760982 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 760985 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 760986 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 760991 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 760996 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@5657776 main
> [junit4:junit4] 2> 760996 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 760997 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5657776 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 760998 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667677151232)],commit=} 0 9
> [junit4:junit4] 2> 761009 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 2
> [junit4:junit4] 2> 761010 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:36101/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 7
> [junit4:junit4] 2> 761010 T2149 C994 P36101 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 761013 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761013 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 761014 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 761014 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 761015 T2152 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@4ba28ccb main
> [junit4:junit4] 2> 761015 T2199 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@2d76073e main
> [junit4:junit4] 2> 761016 T2194 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 761017 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ba28ccb main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 761017 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d76073e main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 761016 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761016 T2198 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 761016 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761018 T2198 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 761018 T2199 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 761017 T2194 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 761018 T2152 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
> [junit4:junit4] 2> 761020 T2198 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@23594073 main
> [junit4:junit4] 2> 761021 T2194 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@3b4fcbaa main
> [junit4:junit4] 2> 761021 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761021 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23594073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 761021 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 761022 T2198 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
> [junit4:junit4] 2> 761022 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b4fcbaa main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
> [junit4:junit4] 2> 761023 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
> [junit4:junit4] 2> 761024 T2149 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
> [junit4:junit4] 2> 761025 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761027 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761030 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761033 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761035 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761038 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761041 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761042 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761044 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761049 T2153 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761050 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761053 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761054 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761060 T2154 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761061 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761064 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761065 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761066 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 761073 T2200 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761075 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761079 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761080 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 761085 T2201 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 761086 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 761089 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 761090 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 761098 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1 (1439363667787251712)]} 0 4
> [junit4:junit4] 2> 761111 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 3
> [junit4:junit4] 2> 761112 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 9
> [junit4:junit4] 2> 761112 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1]} 0 12
> [junit4:junit4] 2> 761116 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667809271808)]} 0 1
> [junit4:junit4] 2> 761128 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 3
> [junit4:junit4] 2> 761128 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 9
> [junit4:junit4] 2> 761132 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[3 (1439363667826049024)]} 0 1
> [junit4:junit4] 2> 761140 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439363667830243328)]} 0 1
> [junit4:junit4] 2> 761141 T2119 C995 P4573
>
> [...truncated too long message...]
>
> TestBloomFilteredLucene41Postings), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=Memory doPackFST= true), foo_i=PostingsFormat(name=SimpleText), regex_dup_B_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_dt1=PostingsFormat(name=SimpleText), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti1=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=SimpleText), val_i=PostingsFormat(name=Memory doPackFST= true), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), series_t=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tdt1=PostingsFormat(name=SimpleText), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=SimpleText), n_l1=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=PostingsFormat(name=SimpleText), inStock=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_CO, timezone=America/Bahia
> [junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=180932536,total=536870912
> [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestBadConfig, TestPseudoReturnFields, TestIBSimilarityFactory, TestReplicationHandler, DocValuesTest, FileBasedSpellCheckerTest, BasicHdfsTest, PrimUtilsTest, TestPHPSerializedResponseWriter, TestLRUCache, SOLR749Test, ParsingFieldUpdateProcessorsTest, TestAddFieldRealTimeGet, DistributedTermsComponentTest, TestMaxScoreQueryParser, LoggingHandlerTest, ResourceLoaderTest, TestStressVersions, MinimalSchemaTest, TestDefaultSearchFieldResource, RegexBoostProcessorTest, TestFunctionQuery, PolyFieldTest, TestSolrDeletionPolicy1, ShowFileRequestHandlerTest, TestCopyFieldCollectionResource, TestQueryUtils, TestLFUCache, PrimitiveFieldTypeTest, TestTrie, TestSearchPerf, SuggesterTest, QueryEqualityTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BlockDirectoryTest, SolrIndexSplitterTest, SyncSliceTest, BadCopyFieldTest, DirectUpdateHandlerOptimizeTest, SynonymTokenizerTest, TestManagedSchemaFieldResource, SuggesterFSTTest, PeerSyncTest, TestSolrDeletionPolicy2, TestGroupingSearch, TestDynamicFieldCollectionResource, AutoCommitTest, TestRandomDVFaceting, ShardRoutingTest, OverseerCollectionProcessorTest, TestFieldResource, DistributedSpellCheckComponentTest, TestSolrIndexConfig, CSVRequestHandlerTest, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, TestXIncludeConfig, TestJoin, TestStressRecovery, TestWriterPerf, BasicDistributedZkTest]
> [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=61B7523138BE4F06 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=America/Bahia -Dtests.file.encoding=ISO-8859-1
> [junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
> [junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> [junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> [junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
> [junit4:junit4] Completed on J0 in 242.77s, 1 test, 2 errors <<< FAILURES!
>
> [...truncated 533 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 628 ignored (5 assumptions)
>
> Total time: 40 minutes 55 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4102 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4102/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
Build Log:
[...truncated 9599 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 732978 T2067 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zm/x
[junit4:junit4] 2> 732981 T2067 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372684133825
[junit4:junit4] 2> 732983 T2067 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 732983 T2068 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 733084 T2067 oasc.ZkTestServer.run start zk server on port:17602
[junit4:junit4] 2> 733085 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733090 T2074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3329a45 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733090 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733091 T2067 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 733108 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733110 T2076 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4792a0c9 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733110 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733111 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 733115 T2067 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 733118 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 733121 T2067 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 733125 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 733125 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 733131 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 733132 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 733240 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 733240 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 733245 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 733246 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 733249 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 733250 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 733257 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 733258 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 733261 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 733262 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 733265 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 733265 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 733273 T2067 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 733273 T2067 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 733458 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 733461 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:19896
[junit4:junit4] 2> 733462 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 733462 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/solr.xml
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.<init> New CoreContainer 1845363899
[junit4:junit4] 2> 733463 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
[junit4:junit4] 2> 733464 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/'
[junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 733550 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 733551 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 733552 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 733553 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 733553 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 733564 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 733565 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 733565 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 733566 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733569 T2087 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5766e024 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733569 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733584 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 733597 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 733599 T2089 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6974bbb6 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 733599 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 733602 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 733611 T2067 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 733614 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 733621 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:19896_zm%2Fx
[junit4:junit4] 2> 733623 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:19896_zm%2Fx
[junit4:junit4] 2> 733627 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 733642 T2067 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 733646 T2067 oasc.Overseer.start Overseer (id=89960227394879491-127.0.0.1:19896_zm%2Fx-n_0000000000) starting
[junit4:junit4] 2> 733656 T2067 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 733667 T2091 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 733667 T2067 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 733670 T2067 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 733677 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 733680 T2090 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 733684 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 733685 T2092 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 735184 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 735184 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
[junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 735185 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 735201 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 735688 T2092 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 735688 T2092 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1
[junit4:junit4] 2> 735689 T2092 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 735690 T2092 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 735690 T2092 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 735692 T2092 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/'
[junit4:junit4] 2> 735693 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/README' to classloader
[junit4:junit4] 2> 735694 T2092 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 735746 T2092 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 735808 T2092 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 735911 T2092 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 735918 T2092 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 736551 T2092 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 736564 T2092 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 736568 T2092 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 736585 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 736590 T2092 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 736595 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 736596 T2092 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 736597 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 736598 T2092 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 736598 T2092 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 736598 T2092 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372684134121/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/
[junit4:junit4] 2> 736599 T2092 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 736600 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data
[junit4:junit4] 2> 736600 T2092 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index/
[junit4:junit4] 2> 736600 T2092 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 736601 T2092 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/control/data/index
[junit4:junit4] 2> 736605 T2092 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> 736606 T2092 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 736608 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 736609 T2092 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 736610 T2092 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 736611 T2092 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 736612 T2092 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 736628 T2092 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 736636 T2092 oass.SolrIndexSearcher.<init> Opening Searcher@782f5cc2 main
[junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 736637 T2092 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 736641 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@782f5cc2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 736642 T2092 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 736643 T2092 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:19896/zm/x collection:control_collection shard:shard1
[junit4:junit4] 2> 736643 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 736662 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 736665 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 736665 T2092 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:19896/zm/x/collection1/
[junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 736666 T2092 oasc.SyncStrategy.syncToMe http://127.0.0.1:19896/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 736666 T2092 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:19896/zm/x/collection1/ shard1
[junit4:junit4] 2> 736667 T2092 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 736707 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 736715 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.register We are http://127.0.0.1:19896/zm/x/collection1/ and leader is http://127.0.0.1:19896/zm/x/collection1/
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:19896/zm/x
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 736724 T2092 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 736725 T2092 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 736726 T2092 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 736728 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 736729 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 736746 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 736748 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 736750 T2096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78c4fabc name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 736750 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 736752 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 736755 T2067 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 737034 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 737037 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16933
[junit4:junit4] 2> 737038 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 737039 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598
[junit4:junit4] 2> 737040 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/solr.xml
[junit4:junit4] 2> 737040 T2067 oasc.CoreContainer.<init> New CoreContainer 1071384450
[junit4:junit4] 2> 737041 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
[junit4:junit4] 2> 737041 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/'
[junit4:junit4] 2> 737171 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 737172 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 737173 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 737174 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 737175 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 737176 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 737193 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 737194 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 737194 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 737195 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 737198 T2107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14fa8922 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 737199 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 737214 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 737227 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 737229 T2109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66491c84 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 737229 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 737241 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 738222 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 738223 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:19896_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:19896/zm/x"}
[junit4:junit4] 2> 738245 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738245 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738245 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738247 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16933_zm%2Fx
[junit4:junit4] 2> 738256 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16933_zm%2Fx
[junit4:junit4] 2> 738260 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738260 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 738260 T2096 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738261 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 738262 T2109 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738262 T2089 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 738269 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 738273 T2110 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 739760 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 739761 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 739781 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 739781 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 739781 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 740276 T2110 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 740276 T2110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1
[junit4:junit4] 2> 740276 T2110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 740277 T2110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 740277 T2110 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 740279 T2110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/'
[junit4:junit4] 2> 740280 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/README' to classloader
[junit4:junit4] 2> 740281 T2110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 740333 T2110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 740395 T2110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 740497 T2110 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 740504 T2110 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 741168 T2110 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 741181 T2110 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 741185 T2110 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 741202 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741207 T2110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 741211 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741213 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741214 T2110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 741215 T2110 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 741215 T2110 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 741216 T2110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372684137598/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/
[junit4:junit4] 2> 741216 T2110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 741217 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1
[junit4:junit4] 2> 741217 T2110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index/
[junit4:junit4] 2> 741217 T2110 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 741218 T2110 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty1/index
[junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 741221 T2110 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 741224 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 741225 T2110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 741226 T2110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 741227 T2110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 741228 T2110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 741243 T2110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 741250 T2110 oass.SolrIndexSearcher.<init> Opening Searcher@68a89d83 main
[junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 741251 T2110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 741255 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a89d83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 741257 T2110 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 741257 T2110 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16933/zm/x collection:collection1 shard:shard1
[junit4:junit4] 2> 741258 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 741278 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 741281 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 741281 T2110 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 741282 T2110 oasc.SyncStrategy.syncToMe http://127.0.0.1:16933/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 741282 T2110 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16933/zm/x/collection1/ shard1
[junit4:junit4] 2> 741282 T2110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 741286 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741295 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741295 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741295 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.register We are http://127.0.0.1:16933/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16933/zm/x
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 741341 T2110 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 741342 T2110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 741344 T2110 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 741345 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 741346 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 741346 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 741609 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 741612 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45733
[junit4:junit4] 2> 741613 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 741613 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 741614 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204
[junit4:junit4] 2> 741614 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/solr.xml
[junit4:junit4] 2> 741614 T2067 oasc.CoreContainer.<init> New CoreContainer 2090646828
[junit4:junit4] 2> 741615 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
[junit4:junit4] 2> 741615 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/'
[junit4:junit4] 2> 741736 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 741737 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 741738 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 741739 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 741740 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 741741 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 741755 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 741756 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 741756 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 741758 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741761 T2123 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd83276 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741761 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741779 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 741789 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 741792 T2125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c940331 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 741792 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 741798 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 742801 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45733_zm%2Fx
[junit4:junit4] 2> 742802 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 742821 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45733_zm%2Fx
[junit4:junit4] 2> 742821 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:16933_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:16933/zm/x"}
[junit4:junit4] 2> 742863 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 742863 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 742863 T2125 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742863 T2096 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742864 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742882 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742882 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742922 T2109 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742923 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742926 T2089 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 742927 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 742928 T2126 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 742929 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 742957 T2126 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 744428 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 744429 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 744450 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744450 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 744959 T2126 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 744959 T2126 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1
[junit4:junit4] 2> 744959 T2126 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 744960 T2126 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 744960 T2126 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 744962 T2126 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/'
[junit4:junit4] 2> 744963 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/README' to classloader
[junit4:junit4] 2> 744964 T2126 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 745016 T2126 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 745078 T2126 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 745181 T2126 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 745188 T2126 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 745851 T2126 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 745863 T2126 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 745867 T2126 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 745883 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745888 T2126 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 745892 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745894 T2126 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 745896 T2126 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 745896 T2126 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 745896 T2126 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372684142204/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/
[junit4:junit4] 2> 745897 T2126 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 745898 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2
[junit4:junit4] 2> 745899 T2126 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index/
[junit4:junit4] 2> 745899 T2126 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 745899 T2126 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty2/index
[junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
[junit4:junit4] 2> 745903 T2126 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 745906 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 745907 T2126 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 745907 T2126 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 745908 T2126 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 745909 T2126 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 745924 T2126 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 745931 T2126 oass.SolrIndexSearcher.<init> Opening Searcher@3d1a7e9d main
[junit4:junit4] 2> 745932 T2126 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 745933 T2126 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 745937 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d1a7e9d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 745939 T2126 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 745939 T2126 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45733/zm/x collection:collection1 shard:shard2
[junit4:junit4] 2> 745940 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 745958 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 745962 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 745962 T2126 oasc.SyncStrategy.syncToMe http://127.0.0.1:45733/zm/x/collection1/ has no replicas
[junit4:junit4] 2> 745963 T2126 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45733/zm/x/collection1/ shard2
[junit4:junit4] 2> 745963 T2126 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 747462 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 747480 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747480 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.register We are http://127.0.0.1:45733/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45733/zm/x
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 747506 T2126 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 747507 T2126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 747508 T2126 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 747510 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 747511 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 747798 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 747801 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45777
[junit4:junit4] 2> 747802 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 747802 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 747803 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371
[junit4:junit4] 2> 747803 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/solr.xml
[junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.<init> New CoreContainer 805132591
[junit4:junit4] 2> 747804 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
[junit4:junit4] 2> 747805 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/'
[junit4:junit4] 2> 747936 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 747937 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 747938 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 747939 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 747940 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 747941 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 747958 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 747959 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 747960 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 747961 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 747993 T2139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616dff74 name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 747993 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 748027 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 748037 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 748040 T2141 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46e38d21 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 748040 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 748078 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 748987 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 748988 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45733_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45733/zm/x"}
[junit4:junit4] 2> 749008 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749008 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749082 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45777_zm%2Fx
[junit4:junit4] 2> 749084 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45777_zm%2Fx
[junit4:junit4] 2> 749087 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749087 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749087 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 749088 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 749088 T2125 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749089 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 749089 T2109 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749090 T2089 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749096 T2141 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 749102 T2142 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 749104 T2142 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 750514 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 750515 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 750533 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 750533 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 751106 T2142 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 751106 T2142 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1
[junit4:junit4] 2> 751106 T2142 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 751107 T2142 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 751107 T2142 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 751109 T2142 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/'
[junit4:junit4] 2> 751110 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 751111 T2142 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/lib/README' to classloader
[junit4:junit4] 2> 751163 T2142 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 751225 T2142 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 751328 T2142 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 751335 T2142 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 751970 T2142 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 751982 T2142 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 751987 T2142 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 752003 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 752008 T2142 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 752013 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 752014 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 752015 T2142 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 752016 T2142 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 752017 T2142 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 752017 T2142 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372684148371/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/
[junit4:junit4] 2> 752017 T2142 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 752018 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3
[junit4:junit4] 2> 752019 T2142 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index/
[junit4:junit4] 2> 752019 T2142 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 752020 T2142 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty3/index
[junit4:junit4] 2> 752023 T2142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 752024 T2142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 752026 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 752027 T2142 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 752028 T2142 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 752029 T2142 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 752030 T2142 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 752047 T2142 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 752054 T2142 oass.SolrIndexSearcher.<init> Opening Searcher@70a83d7a main
[junit4:junit4] 2> 752055 T2142 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 752056 T2142 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 752060 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a83d7a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 752062 T2142 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 752062 T2142 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45777/zm/x collection:collection1 shard:shard1
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.register We are http://127.0.0.1:45777/zm/x/collection1/ and leader is http://127.0.0.1:16933/zm/x/collection1/
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45777/zm/x
[junit4:junit4] 2> 752066 T2142 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 752067 T2142 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C991 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C991_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 752067 T2144 C991 P45777 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 752068 T2142 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 752068 T2144 C991 P45777 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 752069 T2144 C991 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 752069 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 752070 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 752070 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 752071 T2144 C991 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 752096 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 752360 T2067 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 752363 T2067 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36101
[junit4:junit4] 2> 752364 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 752365 T2067 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938
[junit4:junit4] 2> 752366 T2067 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/solr.xml
[junit4:junit4] 2> 752366 T2067 oasc.CoreContainer.<init> New CoreContainer 1659676005
[junit4:junit4] 2> 752367 T2067 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
[junit4:junit4] 2> 752367 T2067 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/'
[junit4:junit4] 2> 752496 T2067 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 752497 T2067 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 752498 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 752499 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 752500 T2067 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 752501 T2067 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 752501 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 752518 T2067 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 752519 T2067 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:17602/solr
[junit4:junit4] 2> 752520 T2067 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 752521 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752524 T2156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cf00b8d name:ZooKeeperConnection Watcher:127.0.0.1:17602 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752525 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752539 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 752552 T2067 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 752554 T2158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a15d897 name:ZooKeeperConnection Watcher:127.0.0.1:17602/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 752555 T2067 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 752566 T2067 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 753541 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 753542 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 753550 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753550 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753571 T2067 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36101_zm%2Fx
[junit4:junit4] 2> 753573 T2067 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36101_zm%2Fx
[junit4:junit4] 2> 753576 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753576 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 753576 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 753576 T2125 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753577 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 753578 T2109 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753578 T2089 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753579 T2141 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753579 T2158 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 753585 T2159 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 753587 T2159 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 754099 T2099 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 754099 T2099 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:45777_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 755065 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 755066 T2090 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 755072 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755072 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 755590 T2159 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 755590 T2159 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1
[junit4:junit4] 2> 755590 T2159 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 755591 T2159 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 755592 T2159 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 755593 T2159 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/'
[junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/README' to classloader
[junit4:junit4] 2> 755595 T2159 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 755647 T2159 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 755709 T2159 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 755812 T2159 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 755819 T2159 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C992 name=collection1 org.apache.solr.core.SolrCore@53b2b3a6 url=http://127.0.0.1:45777/zm/x/collection1 node=127.0.0.1:45777_zm%2Fx C992_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 756102 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:16933/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 756102 T2144 C992 P45777 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:45777/zm/x START replicas=[http://127.0.0.1:16933/zm/x/collection1/] nUpdates=100
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 756103 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 756104 T2144 C992 P45777 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:16933/zm/x/collection1/. core=collection1
[junit4:junit4] 2> 756105 T2144 C992 P45777 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C993 name=collection1 org.apache.solr.core.SolrCore@215eb2d8 url=http://127.0.0.1:16933/zm/x/collection1 node=127.0.0.1:16933_zm%2Fx C993_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:16933_zm%2Fx, base_url=http://127.0.0.1:16933/zm/x, leader=true}
[junit4:junit4] 2> 756113 T2100 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 756116 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 756119 T2101 C993 P16933 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 756120 T2101 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 756120 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 756121 T2101 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@4d16c784 realtime
[junit4:junit4] 2> 756121 T2101 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 756122 T2101 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 756122 T2144 C992 P45777 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 756123 T2144 C992 P45777 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 756125 T2102 C993 P16933 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 756125 T2102 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 756126 T2144 C992 P45777 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 756128 T2144 C992 P45777 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 756453 T2159 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 756466 T2159 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 756470 T2159 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 756487 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756492 T2159 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 756496 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756498 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756499 T2159 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 756500 T2159 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 756500 T2159 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 756501 T2159 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372684152938/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/
[junit4:junit4] 2> 756501 T2159 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@798e662b
[junit4:junit4] 2> 756502 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4
[junit4:junit4] 2> 756502 T2159 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index/
[junit4:junit4] 2> 756503 T2159 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 756503 T2159 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372684133825/jetty4/index
[junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
[junit4:junit4] 2> 756507 T2159 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 756510 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 756511 T2159 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 756512 T2159 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 756513 T2159 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 756514 T2159 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 756531 T2159 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 756538 T2159 oass.SolrIndexSearcher.<init> Opening Searcher@37878dd4 main
[junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 756539 T2159 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 756544 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37878dd4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 756546 T2159 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 756546 T2159 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36101/zm/x collection:collection1 shard:shard2
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.register We are http://127.0.0.1:36101/zm/x/collection1/ and leader is http://127.0.0.1:45733/zm/x/collection1/
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36101/zm/x
[junit4:junit4] 2> 756550 T2159 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 756550 T2159 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C994 name=collection1 org.apache.solr.core.SolrCore@7f9607f7 url=http://127.0.0.1:36101/zm/x/collection1 node=127.0.0.1:36101_zm%2Fx C994_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 756551 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 756551 T2159 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 756552 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 756553 T2067 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 756553 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756554 T2067 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 756572 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756573 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 756574 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 756575 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756585 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 756585 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45777_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45777/zm/x"}
[junit4:junit4] 2> 756590 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 756601 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756602 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756601 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 756601 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 757575 T2115 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 757575 T2115 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:36101_zm%252Fx&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 757578 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 758581 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 759578 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:45733/zm/x/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 759578 T2162 C994 P36101 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36101/zm/x START replicas=[http://127.0.0.1:45733/zm/x/collection1/] nUpdates=100
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 759579 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 759580 T2162 C994 P36101 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 759580 T2162 C994 P36101 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:45733/zm/x/collection1/. core=collection1
[junit4:junit4] 2> 759581 T2162 C994 P36101 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 759584 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C995 name=collection1 org.apache.solr.core.SolrCore@1e8d62e0 url=http://127.0.0.1:45733/zm/x/collection1 node=127.0.0.1:45733_zm%2Fx C995_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45733_zm%2Fx, base_url=http://127.0.0.1:45733/zm/x, leader=true}
[junit4:junit4] 2> 759588 T2116 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 759592 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 759594 T2117 C995 P45733 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@63f3a140 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a20d579),segFN=segments_1,generation=1}
[junit4:junit4] 2> 759596 T2117 C995 P45733 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 759596 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 759597 T2117 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@496ac93a realtime
[junit4:junit4] 2> 759597 T2117 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 759597 T2117 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 759598 T2162 C994 P36101 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 759598 T2162 C994 P36101 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 759600 T2118 C995 P45733 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 759600 T2118 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 759601 T2162 C994 P36101 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 759602 T2162 C994 P36101 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 759639 T2162 C994 P36101 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 759641 T2090 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 759642 T2090 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:36101_zm%2Fx",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36101/zm/x"}
[junit4:junit4] 2> 759653 T2125 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2141 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2089 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2158 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 759653 T2096 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 760586 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760587 T2067 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C996 name=collection1 org.apache.solr.core.SolrCore@565d49dc url=http://127.0.0.1:19896/zm/x/collection1 node=127.0.0.1:19896_zm%2Fx C996_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:19896_zm%2Fx, base_url=http://127.0.0.1:19896/zm/x, leader=true}
[junit4:junit4] 2> 760599 T2079 C996 P19896 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760600 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760602 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@462edf25 realtime
[junit4:junit4] 2> 760602 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667266109440)} 0 6
[junit4:junit4] 2> C994_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:36101_zm%2Fx, base_url=http://127.0.0.1:36101/zm/x}
[junit4:junit4] 2> 760626 T2148 C994 P36101 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3fb835d1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4fdaa1ee),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760627 T2148 C994 P36101 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760629 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@2c01e8e2 realtime
[junit4:junit4] 2> 760629 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667282886656&update.from=http://127.0.0.1:45733/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 4
[junit4:junit4] 2> C992_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:45777_zm%2Fx, base_url=http://127.0.0.1:45777/zm/x}
[junit4:junit4] 2> 760630 T2131 C992 P45777 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 760631 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 760632 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6925ec4e realtime
[junit4:junit4] 2> 760633 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=FROMLEADER&_version_=-1439363667287080960&update.from=http://127.0.0.1:16933/zm/x/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 5
[junit4:junit4] 2> 760634 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439363667287080960)} 0 17
[junit4:junit4] 2> 760634 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439363667282886656)} 0 21
[junit4:junit4] 2> 760636 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760638 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760640 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760642 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760643 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760646 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760648 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760652 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760674 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760675 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760681 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@190c0090 main
[junit4:junit4] 2> 760681 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760682 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@190c0090 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760682 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667322732544)],commit=} 0 31
[junit4:junit4] 2> 760689 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667358384128)]} 0 1
[junit4:junit4] 2> 760690 T2104 C993 P16933 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760692 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760692 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760692 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760693 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760693 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760693 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760695 T2150 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@693e8ddf main
[junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760696 T2121 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@10432ba8 main
[junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
[junit4:junit4] 2> 760697 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@693e8ddf main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760696 T2105 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760696 T2150 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760698 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10432ba8 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760697 T2131 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 760697 T2121 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760699 T2105 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@6a162992 main
[junit4:junit4] 2> 760698 T2150 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760700 T2105 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760700 T2131 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@64c26d35 main
[junit4:junit4] 2> 760700 T2121 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 760701 T2131 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760701 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a162992 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760702 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64c26d35 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 760702 T2105 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760703 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760703 T2104 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667358384128)],commit=} 0 18
[junit4:junit4] 2> 760705 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760707 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760712 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760715 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760717 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760720 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760723 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760724 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760726 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760732 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760732 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760736 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161572&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760737 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760743 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760743 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 760746 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161583&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760747 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760749 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760755 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760755 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760759 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161595&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760760 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760766 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760766 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760769 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161606&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760770 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760775 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760778 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760779 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760780 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@54bd13e5 main
[junit4:junit4] 2> 760781 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760781 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54bd13e5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760782 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667450658816)],commit=} 0 9
[junit4:junit4] 2> 760795 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 2
[junit4:junit4] 2> 760796 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667467436032)]} 0 8
[junit4:junit4] 2> 760796 T2120 C995 P45733 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760798 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760798 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760800 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760800 T2185 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@17b4c540 main
[junit4:junit4] 2> 760800 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760800 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760802 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17b4c540 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760802 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760801 T2185 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760803 T2148 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@67e514d main
[junit4:junit4] 2> 760803 T2133 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760804 T2148 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760804 T2185 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760805 T2103 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
[junit4:junit4] 2> 760805 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67e514d main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760805 T2133 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760807 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 760806 T2103 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 760808 T2133 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@3cd97ef4 main
[junit4:junit4] 2> 760809 T2133 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760809 T2103 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@1ec58f94 main
[junit4:junit4] 2> 760810 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd97ef4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760810 T2103 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760810 T2133 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 760811 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ec58f94 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 760811 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 760812 T2120 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
[junit4:junit4] 2> 760813 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760815 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760819 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760823 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760824 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760828 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760832 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760833 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760835 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760841 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760841 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760846 T2099 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161681&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760847 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 760853 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760853 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760857 T2101 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161693&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760858 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760859 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760866 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760866 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760870 T2177 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161706&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760871 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760877 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760878 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760882 T2181 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161717&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760883 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 760889 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760892 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760893 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760894 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@3b31fd2f main
[junit4:junit4] 2> 760895 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760895 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b31fd2f main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760896 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667570196480)],commit=} 0 9
[junit4:junit4] 2> 760907 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 2
[junit4:junit4] 2> 760908 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45777/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667584876544)]} 0 7
[junit4:junit4] 2> 760908 T2132 C992 P45777 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 760911 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760911 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760912 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760913 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 760913 T2191 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@5dcf10fa main
[junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760914 T2151 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@70a07912 main
[junit4:junit4] 2> 760915 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dcf10fa main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760915 T2187 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
[junit4:junit4] 2> 760916 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70a07912 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 760914 T2134 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760914 T2191 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760916 T2187 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 760915 T2151 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760917 T2134 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@6b4a3fc7 main
[junit4:junit4] 2> 760917 T2191 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 760918 T2134 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760918 T2187 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@5338fea9 main
[junit4:junit4] 2> 760918 T2151 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 760919 T2187 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760919 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b4a3fc7 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760920 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5338fea9 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 760921 T2134 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760921 T2187 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 760922 T2132 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
[junit4:junit4] 2> 760923 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760925 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760929 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760931 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760933 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760936 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760939 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760940 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760942 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760948 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760949 T2115 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760952 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161789&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760953 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760958 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 760959 T2117 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760962 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161799&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760963 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760964 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 760970 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760972 T2178 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760975 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161811&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760976 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 760981 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 760982 T2182 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 760985 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161822&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 760986 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 760991 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@47d81405 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79787b31),segFN=segments_5,generation=5}
[junit4:junit4] 2> 760994 T2079 C996 P19896 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 760996 T2079 C996 P19896 oass.SolrIndexSearcher.<init> Opening Searcher@5657776 main
[junit4:junit4] 2> 760996 T2079 C996 P19896 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 760997 T2093 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5657776 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 760998 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439363667677151232)],commit=} 0 9
[junit4:junit4] 2> 761009 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 2
[junit4:junit4] 2> 761010 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:36101/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667691831296)]} 0 7
[junit4:junit4] 2> 761010 T2149 C994 P36101 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:16933/zm/x/collection1/, StdNode: http://127.0.0.1:45777/zm/x/collection1/, StdNode: http://127.0.0.1:45733/zm/x/collection1/, StdNode: http://127.0.0.1:36101/zm/x/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 761013 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761013 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 761014 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 761014 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 761015 T2152 C994 P36101 oass.SolrIndexSearcher.<init> Opening Searcher@4ba28ccb main
[junit4:junit4] 2> 761015 T2199 C995 P45733 oass.SolrIndexSearcher.<init> Opening Searcher@2d76073e main
[junit4:junit4] 2> 761016 T2194 C993 P16933 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ea62959 lockFactory=org.apache.lucene.store.NativeFSLockFactory@8517ccb),segFN=segments_5,generation=5}
[junit4:junit4] 2> 761017 T2161 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ba28ccb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 761017 T2127 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d76073e main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 761016 T2199 C995 P45733 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761016 T2198 C992 P45777 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@289baade lockFactory=org.apache.lucene.store.NativeFSLockFactory@62203b9),segFN=segments_5,generation=5}
[junit4:junit4] 2> 761016 T2152 C994 P36101 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761018 T2198 C992 P45777 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 761018 T2199 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 761017 T2194 C993 P16933 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 761018 T2152 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 761020 T2198 C992 P45777 oass.SolrIndexSearcher.<init> Opening Searcher@23594073 main
[junit4:junit4] 2> 761021 T2194 C993 P16933 oass.SolrIndexSearcher.<init> Opening Searcher@3b4fcbaa main
[junit4:junit4] 2> 761021 T2198 C992 P45777 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761021 T2143 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23594073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 761021 T2194 C993 P16933 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 761022 T2198 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 761022 T2111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b4fcbaa main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 761023 T2194 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 761024 T2149 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 24
[junit4:junit4] 2> 761025 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761027 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761030 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761033 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761035 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761038 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761041 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761042 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761044 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761049 T2153 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761050 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761053 T2135 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161890&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761054 T2104 C993 P16933 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761060 T2154 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761061 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761064 T2136 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161901&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761065 T2132 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761066 T2067 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 761073 T2200 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761075 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761079 T2137 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161914&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761080 T2120 C995 P45733 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 761085 T2201 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:45733/zm/x/collection1/|127.0.0.1:36101/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 761086 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 761089 T2192 C992 P45777 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={NOW=1372684161926&shard.url=127.0.0.1:16933/zm/x/collection1/|127.0.0.1:45777/zm/x/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 761090 T2149 C994 P36101 oasc.SolrCore.execute [collection1] webapp=/zm/x path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 761098 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1 (1439363667787251712)]} 0 4
[junit4:junit4] 2> 761111 T2131 C992 P45777 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:16933/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 3
[junit4:junit4] 2> 761112 T2103 C993 P16933 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439363667796688896)]} 0 9
[junit4:junit4] 2> 761112 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[1]} 0 12
[junit4:junit4] 2> 761116 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667809271808)]} 0 1
[junit4:junit4] 2> 761128 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 3
[junit4:junit4] 2> 761128 T2119 C995 P45733 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[2 (1439363667813466112)]} 0 9
[junit4:junit4] 2> 761132 T2079 C996 P19896 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={wt=javabin&version=2} {add=[3 (1439363667826049024)]} 0 1
[junit4:junit4] 2> 761140 T2148 C994 P36101 oasup.LogUpdateProcessor.finish [collection1] webapp=/zm/x path=/update params={distrib.from=http://127.0.0.1:45733/zm/x/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439363667830243328)]} 0 1
[junit4:junit4] 2> 761141 T2119 C995 P4573
[...truncated too long message...]
TestBloomFilteredLucene41Postings), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=TestBloomFilteredLucene41Postings), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=Memory doPackFST= true), foo_i=PostingsFormat(name=SimpleText), regex_dup_B_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_dt1=PostingsFormat(name=SimpleText), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti1=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=SimpleText), val_i=PostingsFormat(name=Memory doPackFST= true), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), series_t=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tdt1=PostingsFormat(name=SimpleText), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=PostingsFormat(name=SimpleText), n_l1=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=PostingsFormat(name=SimpleText), inStock=PostingsFormat(name=Asserting)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=es_CO, timezone=America/Bahia
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=180932536,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestBadConfig, TestPseudoReturnFields, TestIBSimilarityFactory, TestReplicationHandler, DocValuesTest, FileBasedSpellCheckerTest, BasicHdfsTest, PrimUtilsTest, TestPHPSerializedResponseWriter, TestLRUCache, SOLR749Test, ParsingFieldUpdateProcessorsTest, TestAddFieldRealTimeGet, DistributedTermsComponentTest, TestMaxScoreQueryParser, LoggingHandlerTest, ResourceLoaderTest, TestStressVersions, MinimalSchemaTest, TestDefaultSearchFieldResource, RegexBoostProcessorTest, TestFunctionQuery, PolyFieldTest, TestSolrDeletionPolicy1, ShowFileRequestHandlerTest, TestCopyFieldCollectionResource, TestQueryUtils, TestLFUCache, PrimitiveFieldTypeTest, TestTrie, TestSearchPerf, SuggesterTest, QueryEqualityTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, BlockDirectoryTest, SolrIndexSplitterTest, SyncSliceTest, BadCopyFieldTest, DirectUpdateHandlerOptimizeTest, SynonymTokenizerTest, TestManagedSchemaFieldResource, SuggesterFSTTest, PeerSyncTest, TestSolrDeletionPolicy2, TestGroupingSearch, TestDynamicFieldCollectionResource, AutoCommitTest, TestRandomDVFaceting, ShardRoutingTest, OverseerCollectionProcessorTest, TestFieldResource, DistributedSpellCheckComponentTest, TestSolrIndexConfig, CSVRequestHandlerTest, FieldMutatingUpdateProcessorTest, SimpleFacetsTest, TestXIncludeConfig, TestJoin, TestStressRecovery, TestWriterPerf, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=61B7523138BE4F06 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CO -Dtests.timezone=America/Bahia -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2538, name=recoveryCmdExecutor-1003-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([61B7523138BE4F06]:0)
[junit4:junit4] Completed on J0 in 242.77s, 1 test, 2 errors <<< FAILURES!
[...truncated 533 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 628 ignored (5 assumptions)
Total time: 40 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4101 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4101/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
Build Log:
[...truncated 9181 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 94969 T521 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 94975 T521 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372638152823
[junit4:junit4] 2> 94976 T521 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 94976 T522 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 95077 T521 oasc.ZkTestServer.run start zk server on port:24079
[junit4:junit4] 2> 95078 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95083 T528 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f762a9e name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95083 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95084 T521 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 95101 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 95102 T530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13d6e17a name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 95103 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 95103 T521 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 95113 T521 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 95117 T521 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 95119 T521 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 95128 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 95129 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 95142 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 95143 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 95247 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 95248 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 95251 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 95252 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 95262 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 95263 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 95265 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 95266 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 95270 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 95270 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 95273 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 95274 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 95277 T521 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 95278 T521 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 96196 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 96460 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24082
[junit4:junit4] 2> 96504 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 96505 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 96505 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135
[junit4:junit4] 2> 96506 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/solr.xml
[junit4:junit4] 2> 96506 T521 oasc.CoreContainer.<init> New CoreContainer 318197175
[junit4:junit4] 2> 96507 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/'
[junit4:junit4] 2> 96507 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/'
[junit4:junit4] 2> 96679 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 96680 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 96680 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 96681 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 96681 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 96682 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 96683 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 96732 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 97215 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 97216 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 97230 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 97231 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 97235 T542 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1164ad0 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 97235 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 97241 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 97255 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 97257 T544 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fd4eeea name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 97258 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 97260 T521 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 97274 T521 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 97278 T521 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 97281 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24082_
[junit4:junit4] 2> 97289 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24082_
[junit4:junit4] 2> 97293 T521 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 97305 T521 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 97315 T521 oasc.Overseer.start Overseer (id=89957213983866883-127.0.0.1:24082_-n_0000000000) starting
[junit4:junit4] 2> 97327 T521 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 97330 T546 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 97331 T521 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 97334 T521 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 97340 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 97342 T545 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 97383 T547 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 97383 T547 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 97385 T547 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 98846 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24082_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24082"}
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 98847 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 98868 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 99387 T547 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 99387 T547 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1
[junit4:junit4] 2> 99388 T547 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 99389 T547 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 99389 T547 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 99392 T547 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/'
[junit4:junit4] 2> 99394 T547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 99395 T547 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/lib/README' to classloader
[junit4:junit4] 2> 99517 T547 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 99604 T547 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 99706 T547 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 99769 T547 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 100565 T547 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 100587 T547 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 100592 T547 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 100625 T547 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 100632 T547 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 100639 T547 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 100640 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 100640 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 100641 T547 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 100642 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 100642 T547 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 100643 T547 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 100643 T547 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372638153135/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/
[junit4:junit4] 2> 100644 T547 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 100646 T547 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data
[junit4:junit4] 2> 100647 T547 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index/
[junit4:junit4] 2> 100647 T547 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 100648 T547 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/control/data/index
[junit4:junit4] 2> 100654 T547 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 100654 T547 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 100659 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 100659 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 100661 T547 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 100662 T547 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 100663 T547 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 100663 T547 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 100664 T547 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 100665 T547 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 100666 T547 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 100683 T547 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 100692 T547 oass.SolrIndexSearcher.<init> Opening Searcher@28031121 main
[junit4:junit4] 2> 100694 T547 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 100694 T547 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 100702 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28031121 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 100707 T547 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 100707 T547 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24082 collection:control_collection shard:shard1
[junit4:junit4] 2> 100719 T547 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 100745 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 100765 T547 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 100768 T547 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 100768 T547 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 100769 T547 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24082/collection1/
[junit4:junit4] 2> 100769 T547 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 100770 T547 oasc.SyncStrategy.syncToMe http://127.0.0.1:24082/collection1/ has no replicas
[junit4:junit4] 2> 100770 T547 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24082/collection1/ shard1
[junit4:junit4] 2> 100771 T547 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 101874 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101923 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 101958 T547 oasc.ZkController.register We are http://127.0.0.1:24082/collection1/ and leader is http://127.0.0.1:24082/collection1/
[junit4:junit4] 2> 101958 T547 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24082
[junit4:junit4] 2> 101959 T547 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 101959 T547 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 101959 T547 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 101961 T547 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 101968 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 101969 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 102069 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 102098 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 102106 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102113 T551 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34ed0665 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102113 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102114 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 102119 T521 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 102391 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 102394 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24086
[junit4:junit4] 2> 102395 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 102396 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 102396 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967
[junit4:junit4] 2> 102397 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/solr.xml
[junit4:junit4] 2> 102398 T521 oasc.CoreContainer.<init> New CoreContainer 1681937453
[junit4:junit4] 2> 102398 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/'
[junit4:junit4] 2> 102399 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/'
[junit4:junit4] 2> 102550 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 102551 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 102552 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 102552 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 102553 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 102553 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 102554 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 102555 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 102555 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 102556 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 102576 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 102577 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 102577 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 102578 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102581 T562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f751b15 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102582 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102599 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 102614 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 102616 T564 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14581b64 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 102617 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 102626 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 103434 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 103436 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24082_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24082"}
[junit4:junit4] 2> 103452 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103452 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103452 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103630 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24086_
[junit4:junit4] 2> 103632 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24086_
[junit4:junit4] 2> 103635 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103635 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 103635 T544 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103636 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 103636 T564 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103644 T551 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 103651 T565 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 103651 T565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 103653 T565 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 104956 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 104957 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24086_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24086"}
[junit4:junit4] 2> 104958 T545 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 104958 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 104977 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 104977 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 104977 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 105655 T565 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 105655 T565 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1
[junit4:junit4] 2> 105656 T565 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 105656 T565 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 105657 T565 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 105659 T565 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/'
[junit4:junit4] 2> 105660 T565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 105661 T565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/lib/README' to classloader
[junit4:junit4] 2> 105741 T565 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 105823 T565 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 105924 T565 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 105966 T565 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 106739 T565 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 106763 T565 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 106768 T565 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 106798 T565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 106804 T565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 106810 T565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 106812 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 106812 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 106813 T565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 106814 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 106814 T565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 106815 T565 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 106815 T565 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372638159967/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/
[junit4:junit4] 2> 106816 T565 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 106818 T565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1
[junit4:junit4] 2> 106818 T565 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index/
[junit4:junit4] 2> 106819 T565 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 106819 T565 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty1/index
[junit4:junit4] 2> 106824 T565 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 106824 T565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 106829 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 106830 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 106831 T565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 106833 T565 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 106834 T565 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 106834 T565 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 106835 T565 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 106836 T565 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 106837 T565 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 106854 T565 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 106863 T565 oass.SolrIndexSearcher.<init> Opening Searcher@6f23c5ef main
[junit4:junit4] 2> 106864 T565 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 106865 T565 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 106875 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6f23c5ef main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 106881 T565 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 106881 T565 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24086 collection:collection1 shard:shard1
[junit4:junit4] 2> 106882 T565 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 106935 T565 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 106938 T565 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 106938 T565 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 106939 T565 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 106939 T565 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 106940 T565 oasc.SyncStrategy.syncToMe http://127.0.0.1:24086/collection1/ has no replicas
[junit4:junit4] 2> 106940 T565 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24086/collection1/ shard1
[junit4:junit4] 2> 106940 T565 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 107983 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 108003 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108003 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108003 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 108017 T565 oasc.ZkController.register We are http://127.0.0.1:24086/collection1/ and leader is http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 108017 T565 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24086
[junit4:junit4] 2> 108017 T565 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 108018 T565 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 108018 T565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 108020 T565 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 108021 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 108022 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 108022 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 108319 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 108322 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24090
[junit4:junit4] 2> 108323 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 108324 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 108325 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890
[junit4:junit4] 2> 108325 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/solr.xml
[junit4:junit4] 2> 108326 T521 oasc.CoreContainer.<init> New CoreContainer 450148949
[junit4:junit4] 2> 108327 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/'
[junit4:junit4] 2> 108327 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/'
[junit4:junit4] 2> 108490 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 108491 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 108491 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 108492 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 108492 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 108493 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 108493 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 108494 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 108495 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 108495 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 108515 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 108516 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 108516 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 108517 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108521 T578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c0e8d95 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 108522 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 108536 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 108551 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108553 T580 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c160110 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 108553 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 108564 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 109508 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 109509 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24086_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24086"}
[junit4:junit4] 2> 109525 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109525 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109568 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24090_
[junit4:junit4] 2> 109570 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24090_
[junit4:junit4] 2> 109573 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 109573 T544 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109574 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 109575 T564 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109575 T551 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109576 T580 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 109582 T581 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 109582 T581 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 109584 T581 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 111029 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 111030 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24090"}
[junit4:junit4] 2> 111031 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 111031 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 111036 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111036 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 111586 T581 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 111586 T581 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1
[junit4:junit4] 2> 111587 T581 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 111587 T581 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 111588 T581 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 111590 T581 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/'
[junit4:junit4] 2> 111591 T581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/lib/README' to classloader
[junit4:junit4] 2> 111592 T581 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 111661 T581 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 111736 T581 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 111838 T581 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 111874 T581 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 112628 T581 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 112649 T581 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 112654 T581 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 112685 T581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 112691 T581 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 112697 T581 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 112698 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 112699 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 112700 T581 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 112701 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 112701 T581 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 112702 T581 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 112702 T581 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372638165890/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/
[junit4:junit4] 2> 112702 T581 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 112704 T581 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2
[junit4:junit4] 2> 112704 T581 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index/
[junit4:junit4] 2> 112705 T581 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 112705 T581 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty2/index
[junit4:junit4] 2> 112711 T581 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d217978 lockFactory=org.apache.lucene.store.NativeFSLockFactory@263a4b0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 112711 T581 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 112716 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 112717 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 112718 T581 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 112719 T581 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 112720 T581 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 112721 T581 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 112721 T581 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 112723 T581 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 112724 T581 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 112740 T581 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 112748 T581 oass.SolrIndexSearcher.<init> Opening Searcher@652eaec main
[junit4:junit4] 2> 112750 T581 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 112750 T581 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 112761 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@652eaec main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 112767 T581 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 112767 T581 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24090 collection:collection1 shard:shard2
[junit4:junit4] 2> 112768 T581 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 112790 T581 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 112801 T581 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 112801 T581 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 112802 T581 oasc.SyncStrategy.syncToMe http://127.0.0.1:24090/collection1/ has no replicas
[junit4:junit4] 2> 112803 T581 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24090/collection1/ shard2
[junit4:junit4] 2> 112803 T581 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 114052 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 114074 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114074 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 114093 T581 oasc.ZkController.register We are http://127.0.0.1:24090/collection1/ and leader is http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 114093 T581 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24090
[junit4:junit4] 2> 114093 T581 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 114094 T581 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 114094 T581 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 114096 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 114097 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 114098 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 114098 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 114388 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 114391 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21017
[junit4:junit4] 2> 114392 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 114393 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 114394 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965
[junit4:junit4] 2> 114394 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/solr.xml
[junit4:junit4] 2> 114395 T521 oasc.CoreContainer.<init> New CoreContainer 1496973014
[junit4:junit4] 2> 114396 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/'
[junit4:junit4] 2> 114396 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/'
[junit4:junit4] 2> 114543 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 114544 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 114544 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 114545 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 114545 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 114546 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 114546 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 114547 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 114548 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 114548 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 114568 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 114569 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 114569 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 114570 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 114573 T594 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c9532e1 name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 114574 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 114577 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 114591 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 114594 T596 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28c61629 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 114594 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 114606 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 115580 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 115581 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24090"}
[junit4:junit4] 2> 115602 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115602 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115610 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21017_
[junit4:junit4] 2> 115612 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21017_
[junit4:junit4] 2> 115614 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115615 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115615 T544 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115615 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 115616 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 115616 T564 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115615 T596 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115617 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 115623 T580 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115624 T551 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 115630 T597 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 115630 T597 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 115632 T597 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 117107 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 117108 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 117109 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 117109 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 117116 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117116 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 117635 T597 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 117635 T597 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1
[junit4:junit4] 2> 117636 T597 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 117636 T597 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 117637 T597 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 117639 T597 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/'
[junit4:junit4] 2> 117640 T597 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 117641 T597 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/lib/README' to classloader
[junit4:junit4] 2> 117705 T597 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 117787 T597 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 117889 T597 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 117921 T597 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 118651 T597 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 118669 T597 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 118673 T597 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 118701 T597 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 118708 T597 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 118713 T597 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 118714 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 118715 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 118716 T597 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 118717 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 118717 T597 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 118717 T597 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 118718 T597 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372638171965/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/
[junit4:junit4] 2> 118718 T597 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 118720 T597 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3
[junit4:junit4] 2> 118720 T597 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index/
[junit4:junit4] 2> 118720 T597 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 118721 T597 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty3/index
[junit4:junit4] 2> 118725 T597 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 118726 T597 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 118730 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 118731 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 118732 T597 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 118733 T597 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 118734 T597 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 118735 T597 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 118735 T597 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 118736 T597 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 118737 T597 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 118752 T597 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 118760 T597 oass.SolrIndexSearcher.<init> Opening Searcher@3a6e99c1 main
[junit4:junit4] 2> 118761 T597 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 118762 T597 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 118772 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a6e99c1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 118778 T597 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 118778 T597 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21017 collection:collection1 shard:shard1
[junit4:junit4] 2> 118782 T597 oasc.ZkController.register We are http://127.0.0.1:21017/collection1/ and leader is http://127.0.0.1:24086/collection1/
[junit4:junit4] 2> 118782 T597 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21017
[junit4:junit4] 2> 118782 T597 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 118783 T597 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C2_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 118790 T599 C2 P21017 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 118790 T597 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 118791 T599 C2 P21017 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 118791 T599 C2 P21017 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 118792 T599 C2 P21017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 118792 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 118793 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 118793 T599 C2 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 118793 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 119165 T521 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 119169 T521 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44764
[junit4:junit4] 2> 119170 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 119170 T521 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 119171 T521 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665
[junit4:junit4] 2> 119171 T521 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/solr.xml
[junit4:junit4] 2> 119172 T521 oasc.CoreContainer.<init> New CoreContainer 341679870
[junit4:junit4] 2> 119173 T521 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/'
[junit4:junit4] 2> 119173 T521 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/'
[junit4:junit4] 2> 119363 T521 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 119364 T521 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 119364 T521 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 119365 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 119365 T521 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 119366 T521 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 119366 T521 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 119367 T521 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 119367 T521 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 119368 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 119391 T521 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 119391 T521 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:24079/solr
[junit4:junit4] 2> 119392 T521 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 119393 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 119396 T611 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46a2ca3e name:ZooKeeperConnection Watcher:127.0.0.1:24079 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 119397 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 119414 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 119431 T521 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 119435 T613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62f5ae30 name:ZooKeeperConnection Watcher:127.0.0.1:24079/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 119435 T521 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 119448 T521 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 120124 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 120125 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 120146 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120146 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120387 T554 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 120388 T554 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4:junit4] 2> 120389 T554 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:21017_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=8
[junit4:junit4] 2> 120451 T521 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44764_
[junit4:junit4] 2> 120453 T521 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44764_
[junit4:junit4] 2> 120455 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120456 T596 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120456 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 120455 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120455 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120455 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 120456 T544 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120459 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 120465 T613 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T580 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T564 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120466 T551 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 120480 T614 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 120480 T614 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 120482 T614 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 121650 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 121652 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 121653 T545 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 121654 T545 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 121689 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 121689 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 122485 T614 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 122486 T614 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1
[junit4:junit4] 2> 122487 T614 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 122488 T614 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 122488 T614 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 122490 T614 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/'
[junit4:junit4] 2> 122492 T614 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 122494 T614 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/lib/README' to classloader
[junit4:junit4] 2> ASYNC NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C3_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 122568 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24086/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 122582 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 122613 T599 C3 P21017 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21017 START replicas=[http://127.0.0.1:24086/collection1/] nUpdates=100
[junit4:junit4] 2> 122613 T614 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 122629 T599 C3 P21017 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 122630 T599 C3 P21017 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 122631 T599 C3 P21017 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 122631 T599 C3 P21017 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24086/collection1/. core=collection1
[junit4:junit4] 2> 122632 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C4_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 122690 T555 C4 P24086 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=9
[junit4:junit4] 2> 122712 T614 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 122731 T556 C4 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 122736 T556 C4 P24086 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 122737 T556 C4 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 122737 T556 C4 P24086 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 122738 T556 C4 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@42a70f8c realtime
[junit4:junit4] 2> 122739 T556 C4 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 122739 T556 C4 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 58
[junit4:junit4] 2> 122813 T614 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 122818 T599 C3 P21017 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 122818 T599 C3 P21017 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 122848 T599 C3 P21017 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 122854 T614 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C5_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 122945 T557 C5 P24086 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 122946 T557 C5 P24086 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=94
[junit4:junit4] 2> ASYNC NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C6_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 122948 T599 C6 P21017 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 122948 T599 C6 P21017 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 122949 T599 C6 P21017 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 122949 T599 C6 P21017 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 122957 T599 C6 P21017 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 123194 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123196 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:21017_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:21017"}
[junit4:junit4] 2> 123223 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123223 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 123702 T614 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 123721 T614 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 123726 T614 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 123757 T614 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 123763 T614 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 123769 T614 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 123771 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 123771 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 123772 T614 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 123773 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 123773 T614 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 123774 T614 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 123774 T614 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372638176665/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/
[junit4:junit4] 2> 123774 T614 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e91824
[junit4:junit4] 2> 123776 T614 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4
[junit4:junit4] 2> 123776 T614 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index/
[junit4:junit4] 2> 123777 T614 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 123777 T614 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372638152822/jetty4/index
[junit4:junit4] 2> 123782 T614 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@74510dba lockFactory=org.apache.lucene.store.NativeFSLockFactory@75c74de9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 123782 T614 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 123787 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 123788 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 123789 T614 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 123791 T614 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 123792 T614 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 123793 T614 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 123800 T614 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 123802 T614 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 123802 T614 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 123819 T614 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 123828 T614 oass.SolrIndexSearcher.<init> Opening Searcher@e9fe846 main
[junit4:junit4] 2> 123829 T614 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 123830 T614 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 123842 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@e9fe846 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 123847 T614 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 123848 T614 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44764 collection:collection1 shard:shard2
[junit4:junit4] 2> 123851 T614 oasc.ZkController.register We are http://127.0.0.1:44764/collection1/ and leader is http://127.0.0.1:24090/collection1/
[junit4:junit4] 2> 123852 T614 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44764
[junit4:junit4] 2> 123852 T614 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 123852 T614 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@182275f2 url=http://127.0.0.1:44764/collection1 node=127.0.0.1:44764_ C7_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 123853 T617 C7 P44764 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 123854 T614 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123854 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 123855 T617 C7 P44764 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 123855 T617 C7 P44764 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 123855 T521 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 123856 T521 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 123857 T617 C7 P44764 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 123857 T521 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 123878 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 123881 T570 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 123881 T521 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 123883 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 124727 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 124728 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 124749 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124750 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124750 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124749 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 124883 T570 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 124883 T570 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:44764_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 124885 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 125887 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 126885 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24090/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 126885 T617 C7 P44764 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44764 START replicas=[http://127.0.0.1:24090/collection1/] nUpdates=100
[junit4:junit4] 2> 126887 T617 C7 P44764 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 126887 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 126888 T617 C7 P44764 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 126889 T617 C7 P44764 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24090/collection1/. core=collection1
[junit4:junit4] 2> 126889 T617 C7 P44764 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 126890 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C8 name=collection1 org.apache.solr.core.SolrCore@6d20036b url=http://127.0.0.1:24090/collection1 node=127.0.0.1:24090_ C8_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24090_, base_url=http://127.0.0.1:24090, leader=true}
[junit4:junit4] 2> 126898 T571 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 126904 T572 C8 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 126908 T572 C8 P24090 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d217978 lockFactory=org.apache.lucene.store.NativeFSLockFactory@263a4b0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 126909 T572 C8 P24090 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 126910 T572 C8 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 126910 T572 C8 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@3a5cd1d3 realtime
[junit4:junit4] 2> 126911 T572 C8 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 126911 T572 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 126913 T617 C7 P44764 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 126913 T617 C7 P44764 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 126917 T573 C8 P24090 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 126918 T573 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 126919 T617 C7 P44764 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 126919 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 126920 T617 C7 P44764 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 126920 T617 C7 P44764 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 126922 T617 C7 P44764 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 127756 T545 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 127757 T545 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44764_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44764"}
[junit4:junit4] 2> 127777 T564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T544 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T596 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T613 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T551 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127777 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 127892 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 127893 T521 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C9 name=collection1 org.apache.solr.core.SolrCore@5dd15fc4 url=http://127.0.0.1:24082/collection1 node=127.0.0.1:24082_ C9_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24082_, base_url=http://127.0.0.1:24082, leader=true}
[junit4:junit4] 2> 128081 T533 C9 P24082 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128082 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128132 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@581f9a8c realtime
[junit4:junit4] 2> 128138 T533 C9 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439315458386821120)} 0 151
[junit4:junit4] 2> C7_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 128210 T604 C7 P44764 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@74510dba lockFactory=org.apache.lucene.store.NativeFSLockFactory@75c74de9),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128211 T604 C7 P44764 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128213 T604 C7 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@3ddf06fd realtime
[junit4:junit4] 2> 128214 T604 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439315458570321920&update.from=http://127.0.0.1:24090/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 8
[junit4:junit4] 2> 128215 T574 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 18
[junit4:junit4] 2> ASYNC NEW_CORE C10 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C10_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 128218 T586 C10 P21017 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 128219 T586 C10 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 128221 T586 C10 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@22d9628f realtime
[junit4:junit4] 2> 128221 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439315458570321920&update.from=http://127.0.0.1:24086/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C11 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C11_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 128222 T558 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439315458570321920)} 0 25
[junit4:junit4] 2> 128224 T603 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 45
[junit4:junit4] 2> 128225 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128227 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128238 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=7
[junit4:junit4] 2> 128243 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128244 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128248 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 128252 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128268 T533 C9 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128327 T533 C9 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128328 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128471 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@1ecd4f54 main
[junit4:junit4] 2> 128473 T533 C9 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128475 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ecd4f54 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128476 T533 C9 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315458639527936)],commit=} 0 219
[junit4:junit4] 2> 128506 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315458867068928)]} 0 4
[junit4:junit4] 2> 128508 T559 C11 P24086 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 128514 T560 C11 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T576 C8 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T606 C7 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128514 T586 C10 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128517 T606 C7 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 128516 T576 C8 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 128518 T606 C7 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@44831670 main
[junit4:junit4] 2> 128519 T576 C8 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@579128ce main
[junit4:junit4] 2> 128520 T606 C7 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128522 T576 C8 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128522 T582 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 128523 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44831670 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 128525 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@579128ce main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 128525 T606 C7 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 128526 T576 C8 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 128529 T560 C11 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128529 T560 C11 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128530 T586 C10 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 128531 T586 C10 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 128533 T560 C11 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@2d62624e main
[junit4:junit4] 2> 128534 T560 C11 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128535 T586 C10 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@2e7c73e6 main
[junit4:junit4] 2> 128536 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d62624e main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128536 T586 C10 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128537 T560 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> 128537 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2e7c73e6 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 128538 T586 C10 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 128540 T559 C11 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315458867068928)],commit=} 0 59
[junit4:junit4] 2> 128541 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128543 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128580 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=33
[junit4:junit4] 2> 128593 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128601 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128606 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 128611 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128612 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128614 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128701 T607 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=77
[junit4:junit4] 2> 128701 T588 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=77
[junit4:junit4] 2> 128740 T588 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186466&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 128741 T559 C11 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=122
[junit4:junit4] 2> 128752 T608 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128752 T589 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128757 T589 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186593&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128759 T587 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128761 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 128772 T609 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128772 T590 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128777 T590 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186613&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128779 T575 C8 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128789 T591 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 128789 T635 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 128794 T591 C10 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186630&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 128796 T605 C7 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 128806 T533 C9 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 128863 T533 C9 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_3,generation=3}
[junit4:junit4] 2> 128864 T533 C9 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 128867 T533 C9 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@c10417 main
[junit4:junit4] 2> ASYNC NEW_CORE C12 name=collection1 org.apache.solr.core.SolrCore@5dd15fc4 url=http://127.0.0.1:24082/collection1 node=127.0.0.1:24082_ C12_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24082_, base_url=http://127.0.0.1:24082, leader=true}
[junit4:junit4] 2> 128879 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 128880 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c10417 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 128882 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459202613248)],commit=} 0 81
[junit4:junit4] 2> ASYNC NEW_CORE C13 name=collection1 org.apache.solr.core.SolrCore@9346193 url=http://127.0.0.1:21017/collection1 node=127.0.0.1:21017_ C13_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:21017_, base_url=http://127.0.0.1:21017}
[junit4:junit4] 2> 129050 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459304325120)]} 0 144
[junit4:junit4] 2> ASYNC NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3c41d377 url=http://127.0.0.1:24086/collection1 node=127.0.0.1:24086_ C14_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24086_, base_url=http://127.0.0.1:24086, leader=true}
[junit4:junit4] 2> 129052 T554 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24090/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459304325120)]} 0 155
[junit4:junit4] 2> ASYNC NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@6d20036b url=http://127.0.0.1:24090/collection1 node=127.0.0.1:24090_ C15_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24090_, base_url=http://127.0.0.1:24090, leader=true}
[junit4:junit4] 2> 129053 T575 C15 P24090 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129059 T592 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C16 name=collection1 org.apache.solr.core.SolrCore@182275f2 url=http://127.0.0.1:44764/collection1 node=127.0.0.1:44764_ C16_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44764_, base_url=http://127.0.0.1:44764}
[junit4:junit4] 2> 129060 T604 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129059 T554 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129059 T570 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129063 T604 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129065 T570 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129065 T604 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@38be4062 main
[junit4:junit4] 2> 129066 T570 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@d8b9fac main
[junit4:junit4] 2> 129066 T604 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129067 T570 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129067 T582 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 129068 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38be4062 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129070 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d8b9fac main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129070 T604 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 129071 T570 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 129075 T592 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 129075 T592 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 129076 T554 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_3,generation=3}
[junit4:junit4] 2> 129076 T554 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 129079 T592 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@2b41cdbc main
[junit4:junit4] 2> 129081 T592 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129081 T554 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@766e5053 main
[junit4:junit4] 2> 129082 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b41cdbc main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 129082 T554 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129083 T592 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 129083 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@766e5053 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 129085 T554 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 129086 T575 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 199
[junit4:junit4] 2> 129087 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129089 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129095 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129101 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129103 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129109 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129113 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129115 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129116 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129125 T572 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129126 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129131 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186968&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129133 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129141 T639 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129142 T589 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129147 T589 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638186984&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129149 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129150 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129159 T640 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129160 T590 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129165 T590 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187002&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129167 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129175 T641 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129176 T591 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129181 T591 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187018&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129183 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129193 T533 C12 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129202 T533 C12 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129202 T533 C12 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129206 T533 C12 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@41de3257 main
[junit4:junit4] 2> 129207 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129208 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41de3257 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129210 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459608412160)],commit=} 0 22
[junit4:junit4] 2> 129232 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459640918016)]} 0 5
[junit4:junit4] 2> 129234 T556 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:21017/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459640918016)]} 0 15
[junit4:junit4] 2> 129235 T587 C13 P21017 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129242 T556 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T647 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T648 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129242 T646 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129244 T648 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129243 T647 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129246 T648 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@1aaa4a39 main
[junit4:junit4] 2> 129246 T647 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@3f3861fb main
[junit4:junit4] 2> 129247 T648 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129248 T647 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129249 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1aaa4a39 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129250 T648 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 129251 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f3861fb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129252 T556 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129252 T647 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 129253 T646 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_4,generation=4}
[junit4:junit4] 2> 129253 T556 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129254 T646 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 129257 T556 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@25fb025 main
[junit4:junit4] 2> 129258 T646 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@531db190 main
[junit4:junit4] 2> 129258 T556 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129260 T646 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129261 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25fb025 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129261 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@531db190 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 129262 T556 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 129264 T646 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> 129265 T587 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 51
[junit4:junit4] 2> 129266 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129268 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129274 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129279 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129281 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129286 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129291 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129293 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129294 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129304 T607 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129304 T649 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129310 T649 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187146&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129312 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 129321 T650 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=2
[junit4:junit4] 2> 129321 T608 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129326 T650 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187163&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129328 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129330 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129339 T651 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129339 T609 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129345 T651 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187181&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 129346 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=12
[junit4:junit4] 2> 129356 T652 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129357 T635 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129362 T652 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187198&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 129364 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 129374 T533 C12 P24082 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129381 T533 C12 P24082 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67fff2dd lockFactory=org.apache.lucene.store.NativeFSLockFactory@69b18bd8),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129382 T533 C12 P24082 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129385 T533 C12 P24082 oass.SolrIndexSearcher.<init> Opening Searcher@6063f086 main
[junit4:junit4] 2> 129386 T533 C12 P24082 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129388 T548 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6063f086 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129390 T533 C12 P24082 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439315459798204416)],commit=} 0 21
[junit4:junit4] 2> 129411 T586 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24086/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439315459829661696)]} 0 5
[junit4:junit4] 2> 129413 T558 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44764/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439315459829661696)]} 0 15
[junit4:junit4] 2> 129414 T605 C16 P44764 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24086/collection1/, StdNode: http://127.0.0.1:21017/collection1/, StdNode: http://127.0.0.1:24090/collection1/, StdNode: http://127.0.0.1:44764/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 129418 T656 C16 P44764 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129418 T558 C14 P24086 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129419 T574 C15 P24090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129418 T655 C13 P21017 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 129420 T574 C15 P24090 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129418 T656 C16 P44764 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 129422 T574 C15 P24090 oass.SolrIndexSearcher.<init> Opening Searcher@4b352dda main
[junit4:junit4] 2> 129423 T656 C16 P44764 oass.SolrIndexSearcher.<init> Opening Searcher@62b1f62 main
[junit4:junit4] 2> 129423 T574 C15 P24090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129424 T656 C16 P44764 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129426 T582 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b352dda main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129426 T616 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@62b1f62 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 129427 T574 C15 P24090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 129428 T558 C14 P24086 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@164cc9b7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6148874d),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129429 T558 C14 P24086 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129427 T656 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 129429 T655 C13 P21017 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2a25a4d7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a32e49c),segFN=segments_5,generation=5}
[junit4:junit4] 2> 129430 T655 C13 P21017 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 129432 T558 C14 P24086 oass.SolrIndexSearcher.<init> Opening Searcher@16364f main
[junit4:junit4] 2> 129433 T558 C14 P24086 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129433 T655 C13 P21017 oass.SolrIndexSearcher.<init> Opening Searcher@cbcb654 main
[junit4:junit4] 2> 129435 T566 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16364f main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129435 T655 C13 P21017 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 129437 T558 C14 P24086 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> 129437 T598 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@cbcb654 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 129439 T655 C13 P21017 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 129440 T605 C16 P44764 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 46
[junit4:junit4] 2> 129441 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129443 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129449 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129454 T587 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 129457 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129463 T575 C15 P24090 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129467 T605 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 129469 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129470 T521 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 129479 T607 C16 P44764 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24090/collection1/|127.0.0.1:44764/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 129480 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 129487 T588 C13 P21017 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372638187321&shard.url=127.0.0.1:24086/collection1/|127.0.0.1:21017/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 129488 T559 C14 P24086 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 statu
[...truncated too long message...]
upt leaked threads:
[junit4:junit4] 2> 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> 231740 T520 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4:junit4] 2> 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_EC, timezone=America/Cayman
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=229778352,total=337641472
[junit4:junit4] 2> NOTE: All tests run in this JVM: [OverseerCollectionProcessorTest, TestXIncludeConfig, TestMaxScoreQueryParser, OverseerTest, FastVectorHighlighterTest, UpdateRequestProcessorFactoryTest, TestSolrXml, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=9340903FD7C7D6D8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Cayman -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=997, name=recoveryCmdExecutor-166-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9340903FD7C7D6D8]:0)
[junit4:junit4] Completed on J0 in 231.86s, 1 test, 2 errors <<< FAILURES!
[...truncated 905 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 930 ignored (1 assumption)
Total time: 43 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4100 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4100/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
Build Log:
[...truncated 9509 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 780741 T1699 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 780746 T1699 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372589296941
[junit4:junit4] 2> 780748 T1699 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 780749 T1700 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 780850 T1699 oasc.ZkTestServer.run start zk server on port:51211
[junit4:junit4] 2> 780851 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780855 T1706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2195d9e0 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780855 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780856 T1699 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 780874 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 780876 T1708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c88d5ab name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 780876 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 780877 T1699 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 780887 T1699 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 780889 T1699 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 780892 T1699 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 780895 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 780895 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 780901 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 780902 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 781006 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781007 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 781010 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 781011 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 781021 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 781021 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 781025 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781026 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 781029 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781029 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 781033 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781033 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 781036 T1699 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781037 T1699 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 781306 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 781310 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:62307
[junit4:junit4] 2> 781311 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 781311 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 781312 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234
[junit4:junit4] 2> 781312 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/solr.xml
[junit4:junit4] 2> 781313 T1699 oasc.CoreContainer.<init> New CoreContainer 205130362
[junit4:junit4] 2> 781314 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/'
[junit4:junit4] 2> 781314 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/'
[junit4:junit4] 2> 781446 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 781447 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 781448 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 781448 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 781449 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 781449 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 781450 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 781450 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 781451 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 781451 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 781468 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 781469 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 781469 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 781470 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781473 T1719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7da2c048 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781474 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781484 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 781495 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 781498 T1721 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@401512e name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 781498 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 781501 T1699 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 781510 T1699 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 781513 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 781520 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:62307_
[junit4:junit4] 2> 781522 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:62307_
[junit4:junit4] 2> 781525 T1699 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 781542 T1699 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 781544 T1699 oasc.Overseer.start Overseer (id=89954012164784131-127.0.0.1:62307_-n_0000000000) starting
[junit4:junit4] 2> 781554 T1699 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 781565 T1723 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 781566 T1699 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 781569 T1699 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 781576 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 781578 T1722 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 781583 T1724 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 781583 T1724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 781584 T1724 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 783083 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 783083 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62307_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62307"}
[junit4:junit4] 2> 783084 T1722 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 783084 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 783101 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 783587 T1724 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 783587 T1724 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1
[junit4:junit4] 2> 783587 T1724 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 783588 T1724 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 783588 T1724 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 783590 T1724 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/'
[junit4:junit4] 2> 783591 T1724 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/lib/README' to classloader
[junit4:junit4] 2> 783592 T1724 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 783646 T1724 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 783708 T1724 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 783809 T1724 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 783816 T1724 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 784446 T1724 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 784460 T1724 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 784464 T1724 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 784482 T1724 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 784488 T1724 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 784492 T1724 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 784494 T1724 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 784495 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 784496 T1724 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 784496 T1724 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 784496 T1724 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372589297234/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/
[junit4:junit4] 2> 784497 T1724 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 784498 T1724 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data
[junit4:junit4] 2> 784498 T1724 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index/
[junit4:junit4] 2> 784498 T1724 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 784499 T1724 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/control/data/index
[junit4:junit4] 2> 784503 T1724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 784504 T1724 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 784507 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 784507 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 784508 T1724 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 784509 T1724 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 784510 T1724 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 784510 T1724 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 784511 T1724 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 784526 T1724 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 784533 T1724 oass.SolrIndexSearcher.<init> Opening Searcher@12124b16 main
[junit4:junit4] 2> 784534 T1724 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 784534 T1724 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 784538 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12124b16 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 784541 T1724 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 784541 T1724 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:62307 collection:control_collection shard:shard1
[junit4:junit4] 2> 784542 T1724 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 784561 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 784564 T1724 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 784564 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:62307/collection1/
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 784565 T1724 oasc.SyncStrategy.syncToMe http://127.0.0.1:62307/collection1/ has no replicas
[junit4:junit4] 2> 784565 T1724 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:62307/collection1/ shard1
[junit4:junit4] 2> 784566 T1724 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 784605 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784623 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.register We are http://127.0.0.1:62307/collection1/ and leader is http://127.0.0.1:62307/collection1/
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:62307
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 784675 T1724 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 784676 T1724 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 784677 T1724 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 784679 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 784679 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 784680 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 784696 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 784698 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 784700 T1728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d4ae61f name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 784700 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 784702 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 784704 T1699 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 784978 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 784980 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10217
[junit4:junit4] 2> 784981 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 784982 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 784982 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898
[junit4:junit4] 2> 784983 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/solr.xml
[junit4:junit4] 2> 784983 T1699 oasc.CoreContainer.<init> New CoreContainer 110884964
[junit4:junit4] 2> 784984 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/'
[junit4:junit4] 2> 784984 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/'
[junit4:junit4] 2> 785117 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 785118 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 785118 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 785119 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 785119 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 785120 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 785120 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 785121 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 785121 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 785122 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 785139 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 785140 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 785141 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 785142 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 785145 T1739 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c706b4d name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 785145 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 785164 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 785177 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 785179 T1741 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cf4bdd9 name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 785179 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 785190 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 786130 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 786131 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:62307_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:62307"}
[junit4:junit4] 2> 786152 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786152 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786152 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786194 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10217_
[junit4:junit4] 2> 786196 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10217_
[junit4:junit4] 2> 786199 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786199 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 786199 T1741 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786200 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 786201 T1721 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786211 T1728 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 786217 T1742 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 786217 T1742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 786218 T1742 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 787657 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 787658 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10217_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10217"}
[junit4:junit4] 2> 787659 T1722 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 787659 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 787675 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 787675 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 787675 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 788221 T1742 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 788221 T1742 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1
[junit4:junit4] 2> 788221 T1742 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 788222 T1742 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 788222 T1742 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 788224 T1742 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/'
[junit4:junit4] 2> 788225 T1742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 788226 T1742 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/lib/README' to classloader
[junit4:junit4] 2> 788280 T1742 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 788342 T1742 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 788444 T1742 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 788452 T1742 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 789074 T1742 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 789088 T1742 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 789091 T1742 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 789109 T1742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 789114 T1742 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 789118 T1742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 789119 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 789120 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 789120 T1742 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 789121 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 789121 T1742 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 789122 T1742 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 789122 T1742 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372589300898/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/
[junit4:junit4] 2> 789122 T1742 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 789123 T1742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1
[junit4:junit4] 2> 789124 T1742 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index/
[junit4:junit4] 2> 789124 T1742 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 789124 T1742 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty1/index
[junit4:junit4] 2> 789128 T1742 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> 789128 T1742 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 789132 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 789132 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 789133 T1742 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 789134 T1742 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 789135 T1742 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 789136 T1742 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 789136 T1742 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 789151 T1742 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 789158 T1742 oass.SolrIndexSearcher.<init> Opening Searcher@4179ae1f main
[junit4:junit4] 2> 789159 T1742 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 789160 T1742 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 789165 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4179ae1f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 789167 T1742 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 789167 T1742 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10217 collection:collection1 shard:shard1
[junit4:junit4] 2> 789168 T1742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 789186 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 789189 T1742 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 789189 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 789190 T1742 oasc.SyncStrategy.syncToMe http://127.0.0.1:10217/collection1/ has no replicas
[junit4:junit4] 2> 789191 T1742 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10217/collection1/ shard1
[junit4:junit4] 2> 789191 T1742 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 790688 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 790785 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790785 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790785 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.register We are http://127.0.0.1:10217/collection1/ and leader is http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10217
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 790829 T1742 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 790830 T1742 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 790866 T1742 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 790867 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 790868 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 790869 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 791151 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 791154 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44467
[junit4:junit4] 2> 791155 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 791155 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 791156 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078
[junit4:junit4] 2> 791156 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/solr.xml
[junit4:junit4] 2> 791157 T1699 oasc.CoreContainer.<init> New CoreContainer 1371466089
[junit4:junit4] 2> 791158 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/'
[junit4:junit4] 2> 791158 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/'
[junit4:junit4] 2> 791290 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 791291 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 791291 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 791292 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 791292 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 791293 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 791293 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 791294 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 791294 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 791295 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 791311 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 791312 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 791312 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 791313 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 791317 T1755 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e5dbba1 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 791317 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 791334 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 791345 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 791347 T1757 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7344adea name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 791347 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 791359 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 792322 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 792323 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10217_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10217"}
[junit4:junit4] 2> 792340 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792340 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792363 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44467_
[junit4:junit4] 2> 792365 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44467_
[junit4:junit4] 2> 792368 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792368 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 792368 T1741 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792368 T1757 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792368 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792369 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 792369 T1728 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792370 T1721 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 792376 T1758 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 792377 T1758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 792378 T1758 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 793845 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 793846 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44467"}
[junit4:junit4] 2> 793847 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 793847 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 793862 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 793862 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 794381 T1758 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 794381 T1758 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1
[junit4:junit4] 2> 794381 T1758 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 794382 T1758 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 794382 T1758 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 794384 T1758 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/'
[junit4:junit4] 2> 794386 T1758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/lib/README' to classloader
[junit4:junit4] 2> 794386 T1758 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 794440 T1758 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 794502 T1758 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 794604 T1758 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 794611 T1758 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 795239 T1758 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 795254 T1758 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 795258 T1758 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 795276 T1758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 795281 T1758 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 795285 T1758 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 795287 T1758 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 795288 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 795288 T1758 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 795289 T1758 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 795289 T1758 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372589307078/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/
[junit4:junit4] 2> 795290 T1758 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 795291 T1758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2
[junit4:junit4] 2> 795291 T1758 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index/
[junit4:junit4] 2> 795292 T1758 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 795292 T1758 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty2/index
[junit4:junit4] 2> 795296 T1758 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@24605c34 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1849901e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 795296 T1758 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 795300 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 795300 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 795301 T1758 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 795302 T1758 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 795303 T1758 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 795304 T1758 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 795305 T1758 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 795321 T1758 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 795328 T1758 oass.SolrIndexSearcher.<init> Opening Searcher@312f8a90 main
[junit4:junit4] 2> 795330 T1758 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 795330 T1758 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 795336 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@312f8a90 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 795339 T1758 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 795339 T1758 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44467 collection:collection1 shard:shard2
[junit4:junit4] 2> 795340 T1758 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 795356 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 795359 T1758 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 795359 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 795360 T1758 oasc.SyncStrategy.syncToMe http://127.0.0.1:44467/collection1/ has no replicas
[junit4:junit4] 2> 795361 T1758 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44467/collection1/ shard2
[junit4:junit4] 2> 795361 T1758 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 795367 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795374 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795374 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.register We are http://127.0.0.1:44467/collection1/ and leader is http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44467
[junit4:junit4] 2> 795418 T1758 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 795419 T1758 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 795419 T1758 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 795441 T1758 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 795443 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 795443 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 795444 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 795733 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 795735 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11586
[junit4:junit4] 2> 795736 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 795737 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 795737 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655
[junit4:junit4] 2> 795738 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/solr.xml
[junit4:junit4] 2> 795738 T1699 oasc.CoreContainer.<init> New CoreContainer 1634567671
[junit4:junit4] 2> 795739 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/'
[junit4:junit4] 2> 795740 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/'
[junit4:junit4] 2> 795878 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 795879 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 795879 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 795880 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 795880 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 795881 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 795881 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 795882 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 795882 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 795883 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 795901 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 795902 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 795902 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 795903 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 795906 T1771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f79fbdb name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 795907 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 795949 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 795961 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 795964 T1773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70ef8558 name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 795964 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 795977 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 796880 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 796881 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:44467_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44467"}
[junit4:junit4] 2> 796902 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796902 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796980 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11586_
[junit4:junit4] 2> 796982 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11586_
[junit4:junit4] 2> 796984 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796984 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 796985 T1757 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1773 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1741 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796985 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796985 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796985 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 796986 T1721 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796986 T1728 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 796999 T1774 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 796999 T1774 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 797001 T1774 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 798408 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 798409 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 798410 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 798410 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 798427 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798427 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798427 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798428 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 798428 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 799003 T1774 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 799003 T1774 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1
[junit4:junit4] 2> 799003 T1774 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 799004 T1774 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 799004 T1774 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 799006 T1774 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/'
[junit4:junit4] 2> 799007 T1774 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/lib/README' to classloader
[junit4:junit4] 2> 799008 T1774 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 799062 T1774 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 799124 T1774 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 799225 T1774 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 799233 T1774 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 799861 T1774 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 799875 T1774 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 799879 T1774 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 799897 T1774 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 799903 T1774 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 799908 T1774 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 799909 T1774 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 799910 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 799911 T1774 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 799911 T1774 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 799911 T1774 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372589311655/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/
[junit4:junit4] 2> 799912 T1774 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 799913 T1774 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3
[junit4:junit4] 2> 799913 T1774 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index/
[junit4:junit4] 2> 799914 T1774 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 799914 T1774 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty3/index
[junit4:junit4] 2> 799918 T1774 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> 799918 T1774 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 799922 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 799922 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 799923 T1774 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 799924 T1774 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 799925 T1774 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 799926 T1774 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 799927 T1774 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 799943 T1774 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 799950 T1774 oass.SolrIndexSearcher.<init> Opening Searcher@6c5dfbd3 main
[junit4:junit4] 2> 799952 T1774 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 799952 T1774 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 799957 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c5dfbd3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 799960 T1774 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 799961 T1774 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11586 collection:collection1 shard:shard1
[junit4:junit4] 2> 799964 T1774 oasc.ZkController.register We are http://127.0.0.1:11586/collection1/ and leader is http://127.0.0.1:10217/collection1/
[junit4:junit4] 2> 799964 T1774 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11586
[junit4:junit4] 2> 799965 T1774 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 799965 T1774 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C335 name=collection1 org.apache.solr.core.SolrCore@4083bcde url=http://127.0.0.1:11586/collection1 node=127.0.0.1:11586_ C335_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 799965 T1776 C335 P11586 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 799966 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 799966 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 799966 T1776 C335 P11586 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 799967 T1776 C335 P11586 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 799967 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 799968 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 799968 T1776 C335 P11586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 799969 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 799995 T1731 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 800261 T1699 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 800264 T1699 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57178
[junit4:junit4] 2> 800264 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 800265 T1699 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 800265 T1699 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187
[junit4:junit4] 2> 800266 T1699 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/solr.xml
[junit4:junit4] 2> 800266 T1699 oasc.CoreContainer.<init> New CoreContainer 1357457359
[junit4:junit4] 2> 800267 T1699 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/'
[junit4:junit4] 2> 800268 T1699 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/'
[junit4:junit4] 2> 800402 T1699 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 800403 T1699 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 800404 T1699 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 800404 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 800405 T1699 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 800405 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 800406 T1699 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 800406 T1699 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 800407 T1699 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 800407 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 800425 T1699 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 800426 T1699 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51211/solr
[junit4:junit4] 2> 800426 T1699 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 800428 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 800455 T1788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6cdb72a5 name:ZooKeeperConnection Watcher:127.0.0.1:51211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 800456 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 800490 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 800503 T1699 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 800508 T1790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c6a43d name:ZooKeeperConnection Watcher:127.0.0.1:51211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 800508 T1699 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 800531 T1699 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 801434 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 801435 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 801458 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801458 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801535 T1699 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57178_
[junit4:junit4] 2> 801537 T1699 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57178_
[junit4:junit4] 2> 801539 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801540 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 801540 T1757 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1773 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801540 T1790 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801540 T1741 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801541 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801541 T1721 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801541 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801541 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 801554 T1728 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 801560 T1791 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 801560 T1791 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 801562 T1791 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 801998 T1731 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 801998 T1731 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:11586_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 802964 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 802965 T1722 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 802986 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 802986 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 803564 T1791 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 803564 T1791 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1
[junit4:junit4] 2> 803564 T1791 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 803565 T1791 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 803566 T1791 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 803567 T1791 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/'
[junit4:junit4] 2> 803569 T1791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 803569 T1791 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/lib/README' to classloader
[junit4:junit4] 2> 803628 T1791 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 803697 T1791 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 803799 T1791 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 803807 T1791 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C335_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10217/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:11586 START replicas=[http://127.0.0.1:10217/collection1/] nUpdates=100
[junit4:junit4] 2> 804000 T1776 C335 P11586 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 804001 T1776 C335 P11586 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10217/collection1/. core=collection1
[junit4:junit4] 2> 804002 T1776 C335 P11586 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C336 name=collection1 org.apache.solr.core.SolrCore@1a24f684 url=http://127.0.0.1:10217/collection1 node=127.0.0.1:10217_ C336_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10217_, base_url=http://127.0.0.1:10217, leader=true}
[junit4:junit4] 2> 804008 T1732 C336 P10217 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 804011 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 804013 T1733 C336 P10217 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> 804014 T1733 C336 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 804014 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 804014 T1733 C336 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@dcc229d realtime
[junit4:junit4] 2> 804015 T1733 C336 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 804015 T1733 C336 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 804016 T1776 C335 P11586 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 804016 T1776 C335 P11586 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 804018 T1734 C336 P10217 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 804018 T1734 C336 P10217 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 804019 T1776 C335 P11586 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 804021 T1776 C335 P11586 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 804331 T1791 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 804341 T1791 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 804344 T1791 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 804357 T1791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 804361 T1791 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 804365 T1791 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 804366 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 804366 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 804367 T1791 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 804367 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 804368 T1791 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 804368 T1791 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 804368 T1791 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372589316187/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/
[junit4:junit4] 2> 804368 T1791 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608532d1
[junit4:junit4] 2> 804369 T1791 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4
[junit4:junit4] 2> 804369 T1791 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index/
[junit4:junit4] 2> 804370 T1791 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 804370 T1791 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372589296940/jetty4/index
[junit4:junit4] 2> 804373 T1791 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118191ef lockFactory=org.apache.lucene.store.NativeFSLockFactory@63f493ec),segFN=segments_1,generation=1}
[junit4:junit4] 2> 804373 T1791 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 804376 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 804376 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 804377 T1791 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 804377 T1791 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 804378 T1791 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 804378 T1791 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 804379 T1791 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 804391 T1791 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 804396 T1791 oass.SolrIndexSearcher.<init> Opening Searcher@59e1bafe main
[junit4:junit4] 2> 804397 T1791 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 804398 T1791 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 804402 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59e1bafe main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 804404 T1791 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 804404 T1791 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57178 collection:collection1 shard:shard2
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.register We are http://127.0.0.1:57178/collection1/ and leader is http://127.0.0.1:44467/collection1/
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57178
[junit4:junit4] 2> 804407 T1791 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 804408 T1791 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C337 name=collection1 org.apache.solr.core.SolrCore@24b9a4f1 url=http://127.0.0.1:57178/collection1 node=127.0.0.1:57178_ C337_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 804408 T1794 C337 P57178 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 804409 T1791 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 804409 T1794 C337 P57178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 804410 T1699 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 804410 T1699 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 804411 T1794 C337 P57178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804411 T1699 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 804424 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804425 T1747 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 804425 T1699 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 804426 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804491 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 804492 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:11586_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:11586"}
[junit4:junit4] 2> 804496 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 804508 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 804508 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 805426 T1747 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 805426 T1747 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:57178_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 805428 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 806430 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C337_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 807428 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44467/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 807428 T1794 C337 P57178 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57178 START replicas=[http://127.0.0.1:44467/collection1/] nUpdates=100
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 807429 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 807430 T1794 C337 P57178 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 807430 T1794 C337 P57178 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44467/collection1/. core=collection1
[junit4:junit4] 2> 807430 T1794 C337 P57178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 807432 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C338 name=collection1 org.apache.solr.core.SolrCore@12f10775 url=http://127.0.0.1:44467/collection1 node=127.0.0.1:44467_ C338_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:44467_, base_url=http://127.0.0.1:44467, leader=true}
[junit4:junit4] 2> 807437 T1748 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 807443 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 807446 T1749 C338 P44467 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@24605c34 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1849901e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 807447 T1749 C338 P44467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 807447 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 807448 T1749 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@6dab7184 realtime
[junit4:junit4] 2> 807448 T1749 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 807448 T1749 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 807449 T1794 C337 P57178 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 807450 T1794 C337 P57178 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 807452 T1750 C338 P44467 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 807452 T1750 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 807453 T1794 C337 P57178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 807455 T1794 C337 P57178 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 807521 T1722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 807522 T1722 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:57178_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:57178"}
[junit4:junit4] 2> 807539 T1757 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807540 T1741 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807540 T1790 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 807539 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 808434 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808435 T1699 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C339 name=collection1 org.apache.solr.core.SolrCore@999168f url=http://127.0.0.1:62307/collection1 node=127.0.0.1:62307_ C339_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:62307_, base_url=http://127.0.0.1:62307, leader=true}
[junit4:junit4] 2> 808446 T1711 C339 P62307 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 808447 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808448 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@1c9aa4d4 realtime
[junit4:junit4] 2> 808449 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439264223669518336)} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C340 name=collection1 org.apache.solr.core.SolrCore@4083bcde url=http://127.0.0.1:11586/collection1 node=127.0.0.1:11586_ C340_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:11586_, base_url=http://127.0.0.1:11586}
[junit4:junit4] 2> 808472 T1763 C340 P11586 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> C337_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:57178_, base_url=http://127.0.0.1:57178}
[junit4:junit4] 2> 808472 T1780 C337 P57178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118191ef lockFactory=org.apache.lucene.store.NativeFSLockFactory@63f493ec),segFN=segments_1,generation=1}
[junit4:junit4] 2> 808472 T1763 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808473 T1780 C337 P57178 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 808475 T1763 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@3b414d20 realtime
[junit4:junit4] 2> 808475 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439264223688392704&update.from=http://127.0.0.1:10217/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223688392704)} 0 6
[junit4:junit4] 2> 808476 T1780 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@147e313a realtime
[junit4:junit4] 2> ASYNC NEW_CORE C341 name=collection1 org.apache.solr.core.SolrCore@1a24f684 url=http://127.0.0.1:10217/collection1 node=127.0.0.1:10217_ C341_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10217_, base_url=http://127.0.0.1:10217, leader=true}
[junit4:junit4] 2> 808476 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223688392704)} 0 15
[junit4:junit4] 2> 808476 T1780 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439264223685246976&update.from=http://127.0.0.1:44467/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439264223685246976)} 0 7
[junit4:junit4] 2> 808477 T1751 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439264223685246976)} 0 19
[junit4:junit4] 2> 808478 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808480 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808483 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808484 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808485 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808487 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808489 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808492 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808495 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808496 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808497 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@6d5f3409 main
[junit4:junit4] 2> 808498 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808498 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d5f3409 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808499 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223719849984)],commit=} 0 8
[junit4:junit4] 2> 808506 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223730335744)]} 0 1
[junit4:junit4] 2> 808506 T1736 C341 P10217 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808508 T1737 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808509 T1763 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808509 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808510 T1753 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@7ed9ef94 main
[junit4:junit4] 2> 808510 T1782 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@60dd3091 main
[junit4:junit4] 2> 808511 T1753 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808512 T1782 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808512 T1759 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808512 T1737 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808512 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@60dd3091 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808513 T1737 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808514 T1782 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 808514 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ed9ef94 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808515 T1753 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808515 T1737 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@3d464b62 main
[junit4:junit4] 2> 808515 T1763 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_2,generation=2}
[junit4:junit4] 2> 808516 T1737 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808516 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d464b62 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808516 T1763 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 808517 T1737 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808519 T1763 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@2c09709a main
[junit4:junit4] 2> 808519 T1763 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808520 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c09709a main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 808520 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 808521 T1736 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223730335744)],commit=} 0 20
[junit4:junit4] 2> 808522 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808524 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808528 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808530 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808532 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808535 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808538 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808539 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808540 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808546 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808546 T1731 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808549 T1731 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324736&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808550 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808555 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808555 T1733 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808559 T1733 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324746&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808560 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808561 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808567 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808567 T1810 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808570 T1810 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324757&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808571 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808577 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 808577 T1813 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808580 T1813 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324767&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808581 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808586 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808589 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808590 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808591 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@18e0af4d main
[junit4:junit4] 2> 808592 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808593 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18e0af4d main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808594 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223817367552)],commit=} 0 10
[junit4:junit4] 2> 808604 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223832047616)]} 0 2
[junit4:junit4] 2> 808605 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:44467/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264223832047616)]} 0 7
[junit4:junit4] 2> 808606 T1752 C338 P44467 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808608 T1765 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808608 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1735 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808610 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808611 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808612 T1817 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@4a850db5 main
[junit4:junit4] 2> 808612 T1780 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@71c4cab0 main
[junit4:junit4] 2> 808612 T1817 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808613 T1780 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808613 T1793 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808613 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a850db5 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808614 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71c4cab0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808615 T1817 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808615 T1780 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808615 T1765 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808616 T1765 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808616 T1735 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_3,generation=3}
[junit4:junit4] 2> 808617 T1735 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 808618 T1765 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@2995be78 main
[junit4:junit4] 2> 808620 T1765 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808620 T1735 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@5e70b175 main
[junit4:junit4] 2> 808621 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2995be78 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808621 T1735 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808622 T1765 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 808622 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e70b175 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 808623 T1735 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 808623 T1752 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
[junit4:junit4] 2> 808624 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808626 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808630 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808632 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808633 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808636 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808639 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808640 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808642 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808647 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808648 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808651 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324838&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808652 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808657 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808658 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808661 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324848&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808662 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808663 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808668 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808669 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808672 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324859&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808673 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808678 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808679 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808682 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324869&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808683 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808688 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808691 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808692 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808693 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@775d6a2 main
[junit4:junit4] 2> 808694 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808695 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@775d6a2 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808695 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264223924322304)],commit=} 0 9
[junit4:junit4] 2> 808708 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264223940050944)]} 0 2
[junit4:junit4] 2> 808709 T1819 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:11586/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264223940050944)]} 0 8
[junit4:junit4] 2> 808710 T1764 C340 P11586 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808713 T1819 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1823 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808713 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808714 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808716 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808716 T1783 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@1ae8b31f main
[junit4:junit4] 2> 808716 T1824 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@4af2ae4d main
[junit4:junit4] 2> 808717 T1783 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808717 T1824 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808718 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ae8b31f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808718 T1819 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808718 T1783 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808718 T1819 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808718 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4af2ae4d main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808720 T1824 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 808720 T1823 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_4,generation=4}
[junit4:junit4] 2> 808720 T1819 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@4d5b490b main
[junit4:junit4] 2> 808720 T1823 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 808721 T1819 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808722 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d5b490b main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808722 T1819 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808723 T1823 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@616388e6 main
[junit4:junit4] 2> 808725 T1823 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808726 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@616388e6 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 808726 T1823 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 808727 T1764 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 808728 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808729 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808733 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808737 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808738 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808741 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808744 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808745 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808746 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808752 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808753 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808755 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324942&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808756 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808761 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808761 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808764 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324952&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808765 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 808766 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808771 T1809 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808772 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808775 T1768 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324962&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808776 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808781 T1814 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808781 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808784 T1769 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589324972&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808785 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 808790 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808793 T1711 C339 P62307 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@261e70ac lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b42994b),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808794 T1711 C339 P62307 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808795 T1711 C339 P62307 oass.SolrIndexSearcher.<init> Opening Searcher@2c33afe0 main
[junit4:junit4] 2> 808796 T1711 C339 P62307 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808797 T1725 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c33afe0 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808797 T1711 C339 P62307 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439264224031277056)],commit=} 0 9
[junit4:junit4] 2> 808810 T1763 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10217/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439264224047005696)]} 0 2
[junit4:junit4] 2> 808810 T1826 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:57178/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439264224047005696)]} 0 7
[junit4:junit4] 2> 808811 T1781 C337 P57178 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10217/collection1/, StdNode: http://127.0.0.1:11586/collection1/, StdNode: http://127.0.0.1:44467/collection1/, StdNode: http://127.0.0.1:57178/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 808814 T1830 C340 P11586 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1826 C341 P10217 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808814 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 808815 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808815 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 808816 T1831 C338 P44467 oass.SolrIndexSearcher.<init> Opening Searcher@b9f002f main
[junit4:junit4] 2> 808816 T1784 C337 P57178 oass.SolrIndexSearcher.<init> Opening Searcher@7847a8fd main
[junit4:junit4] 2> 808817 T1831 C338 P44467 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808817 T1784 C337 P57178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808818 T1830 C340 P11586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@56bb25e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@582dc5aa),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808818 T1793 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:309)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 808818 T1759 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b9f002f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808819 T1793 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7847a8fd main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 808818 T1830 C340 P11586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808819 T1784 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808819 T1831 C338 P44467 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 808819 T1826 C341 P10217 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b2ad8c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@75af8243),segFN=segments_5,generation=5}
[junit4:junit4] 2> 808820 T1830 C340 P11586 oass.SolrIndexSearcher.<init> Opening Searcher@19b09c17 main
[junit4:junit4] 2> 808821 T1826 C341 P10217 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 808821 T1830 C340 P11586 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808822 T1775 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19b09c17 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808823 T1830 C340 P11586 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 808823 T1826 C341 P10217 oass.SolrIndexSearcher.<init> Opening Searcher@2887623d main
[junit4:junit4] 2> 808824 T1826 C341 P10217 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 808825 T1743 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2887623d main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 808825 T1826 C341 P10217 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 808826 T1781 C337 P57178 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 808827 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808829 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808833 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808837 T1764 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 808838 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808841 T1752 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808844 T1781 C337 P57178 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808845 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808847 T1699 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 808853 T1747 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808853 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808856 T1766 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325043&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 808857 T1736 C341 P10217 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 808862 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 808862 T1749 C338 P44467 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:44467/collection1/|127.0.0.1:57178/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 808865 T1767 C340 P11586 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372589325052&shard.url=127.0.0.1:10217/collection1/|127.0.0.1:11586/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[j
[...truncated too long message...]
it4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene41, sim=DefaultSimilarity, locale=bg, timezone=SystemV/PST8PDT
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=132777480,total=456130560
[junit4:junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, SuggesterTest, DocumentBuilderTest, BlockCacheTest, TestDocumentBuilder, TermVectorComponentTest, QueryParsingTest, SystemInfoHandlerTest, FileBasedSpellCheckerTest, BinaryUpdateRequestHandlerTest, RecoveryZkTest, MoreLikeThisHandlerTest, PluginInfoTest, ShardRoutingTest, SimpleFacetsTest, TestPluginEnable, BasicFunctionalityTest, TestFastWriter, TestSweetSpotSimilarityFactory, TestMaxScoreQueryParser, TestCopyFieldCollectionResource, DirectSolrConnectionTest, TestDynamicFieldCollectionResource, TestSolrQueryParser, TestFunctionQuery, WordBreakSolrSpellCheckerTest, ResponseLogComponentTest, CoreAdminHandlerTest, UpdateRequestProcessorFactoryTest, ZkCLITest, TestReplicationHandler, PolyFieldTest, TestSearchPerf, SyncSliceTest, CircularListTest, TestFuzzyAnalyzedSuggestions, UnloadDistributedZkTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=DC0881C3A2CB615A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=SystemV/PST8PDT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2168, name=recoveryCmdExecutor-878-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DC0881C3A2CB615A]:0)
[junit4:junit4] Completed on J0 in 242.04s, 1 test, 2 errors <<< FAILURES!
[...truncated 618 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 811 ignored (4 assumptions)
Total time: 39 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4099 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4099/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([91C3E9011B6A65D8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([91C3E9011B6A65D8]:0)
Build Log:
[...truncated 9582 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 901525 T2868 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 901530 T2868 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372544873050
[junit4:junit4] 2> 901532 T2868 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 901532 T2869 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 901633 T2868 oasc.ZkTestServer.run start zk server on port:23975
[junit4:junit4] 2> 901634 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 901639 T2875 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32429fa2 name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 901639 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 901639 T2868 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 901653 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 901655 T2877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6746687a name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 901656 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 901656 T2868 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 901666 T2868 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 901668 T2868 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 901671 T2868 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 901679 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 901680 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 901691 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 901692 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 901806 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 901807 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 901810 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 901811 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 901814 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 901814 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 901822 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 901823 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 901825 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 901826 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 901829 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 901830 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 901832 T2868 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 901833 T2868 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 902111 T2868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 902117 T2868 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25060
[junit4:junit4] 2> 902118 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 902119 T2868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 902119 T2868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360
[junit4:junit4] 2> 902120 T2868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/solr.xml
[junit4:junit4] 2> 902121 T2868 oasc.CoreContainer.<init> New CoreContainer 159636505
[junit4:junit4] 2> 902121 T2868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/'
[junit4:junit4] 2> 902122 T2868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/'
[junit4:junit4] 2> 902257 T2868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 902258 T2868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 902259 T2868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 902260 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 902260 T2868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 902261 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 902261 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 902262 T2868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 902262 T2868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 902263 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 902281 T2868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 902282 T2868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23975/solr
[junit4:junit4] 2> 902282 T2868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 902283 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 902286 T2888 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1247492b name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 902287 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 902300 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 902313 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 902315 T2890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39797d95 name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 902315 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 902317 T2868 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 902326 T2868 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 902330 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 902337 T2868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25060_
[junit4:junit4] 2> 902338 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25060_
[junit4:junit4] 2> 902341 T2868 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 902358 T2868 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 902366 T2868 oasc.Overseer.start Overseer (id=89951100800729091-127.0.0.1:25060_-n_0000000000) starting
[junit4:junit4] 2> 902375 T2868 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 902387 T2892 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 902388 T2868 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 902390 T2868 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 902392 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 902394 T2891 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 902398 T2893 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 902399 T2893 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 902400 T2893 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 903897 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 903898 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25060_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25060"}
[junit4:junit4] 2> 903898 T2891 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 903899 T2891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 903918 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 904402 T2893 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 904402 T2893 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/collection1
[junit4:junit4] 2> 904403 T2893 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 904403 T2893 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 904404 T2893 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 904405 T2893 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/collection1/'
[junit4:junit4] 2> 904407 T2893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 904407 T2893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/collection1/lib/README' to classloader
[junit4:junit4] 2> 904462 T2893 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 904524 T2893 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 904626 T2893 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 904632 T2893 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 905154 T2069 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 905274 T2893 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 905287 T2893 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 905291 T2893 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 905304 T2893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 905310 T2893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 905314 T2893 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 905315 T2893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 905316 T2893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 905316 T2893 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 905317 T2893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 905317 T2893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 905317 T2893 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 905318 T2893 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372544873360/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/control/data/
[junit4:junit4] 2> 905318 T2893 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b8a774d
[junit4:junit4] 2> 905319 T2893 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/control/data
[junit4:junit4] 2> 905320 T2893 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/control/data/index/
[junit4:junit4] 2> 905320 T2893 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 905320 T2893 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/control/data/index
[junit4:junit4] 2> 905325 T2893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_1,generation=1}
[junit4:junit4] 2> 905326 T2893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 905329 T2893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 905329 T2893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 905330 T2893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 905331 T2893 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 905331 T2893 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 905331 T2893 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 905332 T2893 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 905332 T2893 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 905333 T2893 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 905349 T2893 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 905357 T2893 oass.SolrIndexSearcher.<init> Opening Searcher@538f58c4 main
[junit4:junit4] 2> 905358 T2893 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 905358 T2893 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 905362 T2894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@538f58c4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 905364 T2893 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 905364 T2893 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25060 collection:control_collection shard:shard1
[junit4:junit4] 2> 905365 T2893 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 905396 T2893 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 905399 T2893 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 905399 T2893 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 905399 T2893 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25060/collection1/
[junit4:junit4] 2> 905400 T2893 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 905400 T2893 oasc.SyncStrategy.syncToMe http://127.0.0.1:25060/collection1/ has no replicas
[junit4:junit4] 2> 905400 T2893 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25060/collection1/ shard1
[junit4:junit4] 2> 905401 T2893 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 905421 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 905427 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 905457 T2893 oasc.ZkController.register We are http://127.0.0.1:25060/collection1/ and leader is http://127.0.0.1:25060/collection1/
[junit4:junit4] 2> 905457 T2893 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25060
[junit4:junit4] 2> 905457 T2893 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 905457 T2893 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 905458 T2893 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 905459 T2893 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 905460 T2868 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 905461 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 905462 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 905478 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 905479 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 905481 T2897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7345caf4 name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 905482 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 905483 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 905485 T2868 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 905757 T2868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 905761 T2868 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25772
[junit4:junit4] 2> 905761 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 905762 T2868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 905763 T2868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004
[junit4:junit4] 2> 905763 T2868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/solr.xml
[junit4:junit4] 2> 905764 T2868 oasc.CoreContainer.<init> New CoreContainer 880255368
[junit4:junit4] 2> 905764 T2868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/'
[junit4:junit4] 2> 905765 T2868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/'
[junit4:junit4] 2> 905898 T2868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 905899 T2868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 905899 T2868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 905900 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 905900 T2868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 905901 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 905902 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 905902 T2868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 905903 T2868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 905903 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 905919 T2868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 905920 T2868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23975/solr
[junit4:junit4] 2> 905920 T2868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 905921 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 905924 T2908 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@89c473f name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 905925 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 905942 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 905952 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 905954 T2910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41935c1b name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 905955 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 905959 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 906932 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 906933 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25060_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25060"}
[junit4:junit4] 2> 906950 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 906951 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 906951 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 906963 T2868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25772_
[junit4:junit4] 2> 906965 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25772_
[junit4:junit4] 2> 906968 T2897 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 906968 T2890 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 906968 T2910 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 906968 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 906968 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 906969 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 906982 T2911 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 906982 T2911 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 906984 T2911 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 908084 T2084 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 908455 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 908456 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25772_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25772"}
[junit4:junit4] 2> 908456 T2891 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 908457 T2891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 908461 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 908461 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 908461 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 908986 T2911 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 908986 T2911 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/collection1
[junit4:junit4] 2> 908986 T2911 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 908987 T2911 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 908987 T2911 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 908989 T2911 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/collection1/'
[junit4:junit4] 2> 908990 T2911 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/collection1/lib/README' to classloader
[junit4:junit4] 2> 908991 T2911 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 909045 T2911 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 909108 T2911 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 909209 T2911 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 909215 T2911 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 909858 T2911 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 909871 T2911 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 909876 T2911 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 909889 T2911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 909894 T2911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 909899 T2911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 909900 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 909900 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 909901 T2911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 909902 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 909902 T2911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 909902 T2911 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 909903 T2911 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372544877004/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty1/
[junit4:junit4] 2> 909903 T2911 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b8a774d
[junit4:junit4] 2> 909904 T2911 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty1
[junit4:junit4] 2> 909904 T2911 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty1/index/
[junit4:junit4] 2> 909905 T2911 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 909905 T2911 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty1/index
[junit4:junit4] 2> 909909 T2911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 909909 T2911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 909913 T2911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 909913 T2911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 909914 T2911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 909915 T2911 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 909915 T2911 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 909916 T2911 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 909916 T2911 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 909917 T2911 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 909917 T2911 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 909934 T2911 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 909941 T2911 oass.SolrIndexSearcher.<init> Opening Searcher@85c6cd3 main
[junit4:junit4] 2> 909942 T2911 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 909943 T2911 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 909947 T2912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@85c6cd3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 909949 T2911 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 909949 T2911 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25772 collection:collection1 shard:shard1
[junit4:junit4] 2> 909950 T2911 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 910002 T2911 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 910013 T2911 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 910013 T2911 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 910014 T2911 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25772/collection1/
[junit4:junit4] 2> 910014 T2911 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 910014 T2911 oasc.SyncStrategy.syncToMe http://127.0.0.1:25772/collection1/ has no replicas
[junit4:junit4] 2> 910015 T2911 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25772/collection1/ shard1
[junit4:junit4] 2> 910015 T2911 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 911486 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 911509 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 911509 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 911509 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 911557 T2911 oasc.ZkController.register We are http://127.0.0.1:25772/collection1/ and leader is http://127.0.0.1:25772/collection1/
[junit4:junit4] 2> 911557 T2911 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25772
[junit4:junit4] 2> 911557 T2911 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 911558 T2911 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 911558 T2911 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 911560 T2911 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 911561 T2868 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 911561 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 911562 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 911846 T2868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 911850 T2868 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:64066
[junit4:junit4] 2> 911851 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 911851 T2868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 911852 T2868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099
[junit4:junit4] 2> 911852 T2868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/solr.xml
[junit4:junit4] 2> 911853 T2868 oasc.CoreContainer.<init> New CoreContainer 221693615
[junit4:junit4] 2> 911854 T2868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/'
[junit4:junit4] 2> 911854 T2868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/'
[junit4:junit4] 2> 911987 T2868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 911988 T2868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 911989 T2868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 911990 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 911990 T2868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 911991 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 911991 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 911992 T2868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 911992 T2868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 911993 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 912008 T2868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 912009 T2868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23975/solr
[junit4:junit4] 2> 912010 T2868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 912011 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 912014 T2924 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a65bc1 name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 912015 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 912030 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 912041 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 912043 T2926 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7701c10b name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 912043 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 912055 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 913025 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 913026 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:25772_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:25772"}
[junit4:junit4] 2> 913031 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 913031 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 913031 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 913031 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 913058 T2868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64066_
[junit4:junit4] 2> 913060 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64066_
[junit4:junit4] 2> 913062 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 913063 T2897 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 913063 T2910 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 913063 T2890 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 913063 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 913063 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 913064 T2926 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 913063 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 913074 T2927 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 913075 T2927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 913076 T2927 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 914544 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 914545 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:64066_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:64066"}
[junit4:junit4] 2> 914545 T2891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 914546 T2891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 914550 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 914550 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 914550 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 914550 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 915078 T2927 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 915078 T2927 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/collection1
[junit4:junit4] 2> 915079 T2927 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 915079 T2927 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 915080 T2927 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 915081 T2927 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/collection1/'
[junit4:junit4] 2> 915083 T2927 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 915083 T2927 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/collection1/lib/README' to classloader
[junit4:junit4] 2> 915137 T2927 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 915200 T2927 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 915301 T2927 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 915307 T2927 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 915948 T2927 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 915961 T2927 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 915965 T2927 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 915979 T2927 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 915984 T2927 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 915989 T2927 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 915990 T2927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 915990 T2927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 915990 T2927 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 915991 T2927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 915991 T2927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 915992 T2927 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 915992 T2927 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372544883099/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty2/
[junit4:junit4] 2> 915993 T2927 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b8a774d
[junit4:junit4] 2> 915994 T2927 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty2
[junit4:junit4] 2> 915994 T2927 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty2/index/
[junit4:junit4] 2> 915995 T2927 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 915995 T2927 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty2/index
[junit4:junit4] 2> 915999 T2927 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@28e6db7f lockFactory=org.apache.lucene.store.NativeFSLockFactory@6dda388f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 915999 T2927 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 916003 T2927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 916003 T2927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 916004 T2927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 916005 T2927 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 916005 T2927 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 916006 T2927 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 916006 T2927 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 916007 T2927 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 916007 T2927 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 916024 T2927 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 916031 T2927 oass.SolrIndexSearcher.<init> Opening Searcher@7b8521ed main
[junit4:junit4] 2> 916032 T2927 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 916033 T2927 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 916037 T2928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b8521ed main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 916040 T2927 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 916040 T2927 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:64066 collection:collection1 shard:shard2
[junit4:junit4] 2> 916041 T2927 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 916063 T2927 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 916073 T2927 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 916074 T2927 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 916074 T2927 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:64066/collection1/
[junit4:junit4] 2> 916074 T2927 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 916075 T2927 oasc.SyncStrategy.syncToMe http://127.0.0.1:64066/collection1/ has no replicas
[junit4:junit4] 2> 916075 T2927 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64066/collection1/ shard2
[junit4:junit4] 2> 916075 T2927 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 917565 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 917586 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 917586 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 917586 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 917586 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 917619 T2927 oasc.ZkController.register We are http://127.0.0.1:64066/collection1/ and leader is http://127.0.0.1:64066/collection1/
[junit4:junit4] 2> 917619 T2927 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64066
[junit4:junit4] 2> 917619 T2927 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 917620 T2927 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 917620 T2927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 917621 T2927 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 917623 T2868 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 917623 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 917624 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 917912 T2868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 917916 T2868 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51113
[junit4:junit4] 2> 917917 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 917917 T2868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 917918 T2868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160
[junit4:junit4] 2> 917918 T2868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/solr.xml
[junit4:junit4] 2> 917919 T2868 oasc.CoreContainer.<init> New CoreContainer 1393276120
[junit4:junit4] 2> 917920 T2868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/'
[junit4:junit4] 2> 917920 T2868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/'
[junit4:junit4] 2> 918055 T2868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 918056 T2868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 918056 T2868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 918057 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 918057 T2868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 918058 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 918059 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 918059 T2868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 918060 T2868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 918060 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 918078 T2868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 918078 T2868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23975/solr
[junit4:junit4] 2> 918079 T2868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 918080 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 918083 T2940 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23f09ebd name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 918084 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 918096 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 918107 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 918109 T2942 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1424ed30 name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 918109 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 918118 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 918355 T2317 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 919091 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 919092 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:64066_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:64066"}
[junit4:junit4] 2> 919105 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919105 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919105 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919105 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919105 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919122 T2868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51113_
[junit4:junit4] 2> 919124 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51113_
[junit4:junit4] 2> 919126 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919126 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 919127 T2910 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 919127 T2890 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 919127 T2897 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 919127 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 919128 T2942 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 919127 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 919128 T2926 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 919128 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 919139 T2943 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 919140 T2943 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 919141 T2943 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 920430 T2334 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 920609 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 920610 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51113_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51113"}
[junit4:junit4] 2> 920610 T2891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 920610 T2891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 920635 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 920635 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 920635 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 920635 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 920635 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 920817 T2351 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 921143 T2943 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 921143 T2943 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/collection1
[junit4:junit4] 2> 921144 T2943 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 921144 T2943 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 921145 T2943 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 921146 T2943 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/collection1/'
[junit4:junit4] 2> 921148 T2943 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 921148 T2943 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/collection1/lib/README' to classloader
[junit4:junit4] 2> 921203 T2943 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 921265 T2943 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 921367 T2943 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 921373 T2943 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 921788 T2368 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 922013 T2943 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 922025 T2943 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 922029 T2943 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 922042 T2943 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 922047 T2943 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 922051 T2943 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 922052 T2943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 922053 T2943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 922053 T2943 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 922054 T2943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 922054 T2943 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 922054 T2943 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 922055 T2943 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372544889160/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty3/
[junit4:junit4] 2> 922055 T2943 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b8a774d
[junit4:junit4] 2> 922056 T2943 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty3
[junit4:junit4] 2> 922057 T2943 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty3/index/
[junit4:junit4] 2> 922057 T2943 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 922057 T2943 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty3/index
[junit4:junit4] 2> 922061 T2943 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 922061 T2943 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 922064 T2943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 922065 T2943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 922065 T2943 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 922066 T2943 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 922067 T2943 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 922067 T2943 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 922067 T2943 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 922068 T2943 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 922069 T2943 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 922084 T2943 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 922091 T2943 oass.SolrIndexSearcher.<init> Opening Searcher@70b0fa58 main
[junit4:junit4] 2> 922092 T2943 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 922092 T2943 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 922097 T2944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70b0fa58 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 922099 T2943 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 922099 T2943 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51113 collection:collection1 shard:shard1
[junit4:junit4] 2> 922102 T2943 oasc.ZkController.register We are http://127.0.0.1:51113/collection1/ and leader is http://127.0.0.1:25772/collection1/
[junit4:junit4] 2> 922103 T2943 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51113
[junit4:junit4] 2> 922103 T2943 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 922103 T2943 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C422 name=collection1 org.apache.solr.core.SolrCore@2f115585 url=http://127.0.0.1:51113/collection1 node=127.0.0.1:51113_ C422_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:51113_, base_url=http://127.0.0.1:51113}
[junit4:junit4] 2> 922104 T2945 C422 P51113 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 922104 T2943 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922104 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 922105 T2945 C422 P51113 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 922105 T2945 C422 P51113 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 922105 T2868 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 922106 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 922106 T2945 C422 P51113 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 922107 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 922128 T2900 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 922139 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 922140 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51113_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51113"}
[junit4:junit4] 2> 922154 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 922154 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 922154 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 922154 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 922154 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 922392 T2868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 922395 T2868 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35935
[junit4:junit4] 2> 922396 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 922397 T2868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 922397 T2868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646
[junit4:junit4] 2> 922398 T2868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/solr.xml
[junit4:junit4] 2> 922398 T2868 oasc.CoreContainer.<init> New CoreContainer 475596038
[junit4:junit4] 2> 922399 T2868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/'
[junit4:junit4] 2> 922400 T2868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/'
[junit4:junit4] 2> 922533 T2868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 922534 T2868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 922534 T2868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 922535 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 922535 T2868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 922536 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 922536 T2868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 922537 T2868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 922538 T2868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 922538 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 922555 T2868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 922556 T2868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:23975/solr
[junit4:junit4] 2> 922557 T2868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 922558 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 922561 T2957 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40678d44 name:ZooKeeperConnection Watcher:127.0.0.1:23975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 922562 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 922575 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 922587 T2868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 922589 T2959 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79e74307 name:ZooKeeperConnection Watcher:127.0.0.1:23975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 922589 T2868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 922594 T2868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 923129 T2900 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 923129 T2900 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:51113_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 923597 T2868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35935_
[junit4:junit4] 2> 923599 T2868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35935_
[junit4:junit4] 2> 923602 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 923602 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 923602 T2890 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923602 T2959 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923602 T2910 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923602 T2897 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923603 T2926 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923603 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923603 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923603 T2942 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 923603 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923623 T2960 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 923623 T2960 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 923624 T2960 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 923658 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 923659 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35935_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35935"}
[junit4:junit4] 2> 923659 T2891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 923659 T2891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 923672 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923672 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923672 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923672 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923672 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 923672 T2959 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 924626 T2960 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 924626 T2960 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/collection1
[junit4:junit4] 2> 924627 T2960 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 924627 T2960 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 924628 T2960 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 924629 T2960 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/collection1/'
[junit4:junit4] 2> 924631 T2960 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/collection1/lib/README' to classloader
[junit4:junit4] 2> 924631 T2960 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 924685 T2960 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 924748 T2960 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 924849 T2960 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 924855 T2960 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C422_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:51113_, base_url=http://127.0.0.1:51113}
[junit4:junit4] 2> 925131 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:25772/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 925131 T2945 C422 P51113 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51113 START replicas=[http://127.0.0.1:25772/collection1/] nUpdates=100
[junit4:junit4] 2> 925132 T2945 C422 P51113 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 925132 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 925132 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 925132 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 925133 T2945 C422 P51113 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 925133 T2945 C422 P51113 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:25772/collection1/. core=collection1
[junit4:junit4] 2> 925133 T2945 C422 P51113 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C423 name=collection1 org.apache.solr.core.SolrCore@420a2bf1 url=http://127.0.0.1:25772/collection1 node=127.0.0.1:25772_ C423_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:25772_, base_url=http://127.0.0.1:25772, leader=true}
[junit4:junit4] 2> 925141 T2901 C423 P25772 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 925148 T2902 C423 P25772 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 925151 T2902 C423 P25772 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 925152 T2902 C423 P25772 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 925152 T2902 C423 P25772 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 925153 T2902 C423 P25772 oass.SolrIndexSearcher.<init> Opening Searcher@1c801d5b realtime
[junit4:junit4] 2> 925153 T2902 C423 P25772 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 925154 T2902 C423 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 925155 T2945 C422 P51113 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 925155 T2945 C422 P51113 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 925157 T2903 C423 P25772 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 925158 T2903 C423 P25772 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 925158 T2945 C422 P51113 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 925159 T2945 C422 P51113 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 925159 T2945 C422 P51113 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 925159 T2945 C422 P51113 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> ASYNC NEW_CORE C424 name=collection1 org.apache.solr.core.SolrCore@2f115585 url=http://127.0.0.1:51113/collection1 node=127.0.0.1:51113_ C424_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:51113_, base_url=http://127.0.0.1:51113}
[junit4:junit4] 2> 925206 T2945 C424 P51113 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 925208 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925208 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51113_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51113"}
[junit4:junit4] 2> 925228 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925228 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925228 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925228 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925228 T2959 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925228 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 925497 T2960 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 925510 T2960 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 925514 T2960 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 925527 T2960 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 925532 T2960 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 925537 T2960 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 925538 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 925538 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 925539 T2960 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 925540 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 925540 T2960 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 925540 T2960 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 925541 T2960 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372544893646/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty4/
[junit4:junit4] 2> 925541 T2960 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4b8a774d
[junit4:junit4] 2> 925542 T2960 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty4
[junit4:junit4] 2> 925543 T2960 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty4/index/
[junit4:junit4] 2> 925543 T2960 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 925543 T2960 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372544873050/jetty4/index
[junit4:junit4] 2> 925548 T2960 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@43cc0b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@28c28348),segFN=segments_1,generation=1}
[junit4:junit4] 2> 925548 T2960 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 925551 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 925552 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 925552 T2960 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 925553 T2960 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 925554 T2960 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 925554 T2960 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 925555 T2960 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 925555 T2960 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 925556 T2960 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 925572 T2960 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 925580 T2960 oass.SolrIndexSearcher.<init> Opening Searcher@754ff161 main
[junit4:junit4] 2> 925581 T2960 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 925582 T2960 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 925586 T2962 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@754ff161 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 925588 T2960 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 925588 T2960 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35935 collection:collection1 shard:shard2
[junit4:junit4] 2> 925591 T2960 oasc.ZkController.register We are http://127.0.0.1:35935/collection1/ and leader is http://127.0.0.1:64066/collection1/
[junit4:junit4] 2> 925592 T2960 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35935
[junit4:junit4] 2> 925592 T2960 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 925592 T2960 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C425 name=collection1 org.apache.solr.core.SolrCore@1583f328 url=http://127.0.0.1:35935/collection1 node=127.0.0.1:35935_ C425_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:35935_, base_url=http://127.0.0.1:35935}
[junit4:junit4] 2> 925593 T2963 C425 P35935 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 925593 T2960 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925593 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 925594 T2963 C425 P35935 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 925594 T2963 C425 P35935 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 925594 T2868 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 925595 T2868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 925595 T2963 C425 P35935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 925596 T2868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 925621 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925622 T2916 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 925623 T2868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 925624 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 925945 T2372 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 926625 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 926732 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 926733 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35935_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35935"}
[junit4:junit4] 2> 926783 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926783 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926783 T2959 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926783 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926783 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 926783 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 927625 T2916 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 927625 T2916 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:35935_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 927627 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 928429 T2387 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4:junit4] 2> 928629 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C425_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:35935_, base_url=http://127.0.0.1:35935}
[junit4:junit4] 2> 929627 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:64066/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 929627 T2963 C425 P35935 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35935 START replicas=[http://127.0.0.1:64066/collection1/] nUpdates=100
[junit4:junit4] 2> 929628 T2963 C425 P35935 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 929628 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 929628 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 929629 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 929629 T2963 C425 P35935 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 929629 T2963 C425 P35935 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:64066/collection1/. core=collection1
[junit4:junit4] 2> 929629 T2963 C425 P35935 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 929631 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C426 name=collection1 org.apache.solr.core.SolrCore@78d33d63 url=http://127.0.0.1:64066/collection1 node=127.0.0.1:64066_ C426_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:64066_, base_url=http://127.0.0.1:64066, leader=true}
[junit4:junit4] 2> 929637 T2917 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 929642 T2918 C426 P64066 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 929644 T2918 C426 P64066 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@28e6db7f lockFactory=org.apache.lucene.store.NativeFSLockFactory@6dda388f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 929645 T2918 C426 P64066 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 929646 T2918 C426 P64066 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 929646 T2918 C426 P64066 oass.SolrIndexSearcher.<init> Opening Searcher@b338d3a realtime
[junit4:junit4] 2> 929647 T2918 C426 P64066 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 929647 T2918 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 929648 T2963 C425 P35935 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 929648 T2963 C425 P35935 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 929650 T2919 C426 P64066 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 929651 T2919 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 929651 T2963 C425 P35935 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 929651 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 929652 T2963 C425 P35935 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 929652 T2963 C425 P35935 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 929658 T2963 C425 P35935 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 929790 T2891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 929791 T2891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35935_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35935"}
[junit4:junit4] 2> 929811 T2942 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 929811 T2910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 929811 T2890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 929811 T2959 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 929811 T2926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 929811 T2897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 930633 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930634 T2868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C427 name=collection1 org.apache.solr.core.SolrCore@16cc1fd2 url=http://127.0.0.1:25060/collection1 node=127.0.0.1:25060_ C427_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:25060_, base_url=http://127.0.0.1:25060, leader=true}
[junit4:junit4] 2> 930647 T2880 C427 P25060 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_1,generation=1}
[junit4:junit4] 2> 930648 T2880 C427 P25060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 930650 T2880 C427 P25060 oass.SolrIndexSearcher.<init> Opening Searcher@7e4f1560 realtime
[junit4:junit4] 2> 930651 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439217643330469888)} 0 7
[junit4:junit4] 2> C425_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:35935_, base_url=http://127.0.0.1:35935}
[junit4:junit4] 2> 930677 T2950 C425 P35935 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@43cc0b66 lockFactory=org.apache.lucene.store.NativeFSLockFactory@28c28348),segFN=segments_1,generation=1}
[junit4:junit4] 2> 930678 T2950 C425 P35935 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 930680 T2950 C425 P35935 oass.SolrIndexSearcher.<init> Opening Searcher@6a769232 realtime
[junit4:junit4] 2> 930681 T2950 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439217643359830016&update.from=http://127.0.0.1:64066/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439217643359830016)} 0 6
[junit4:junit4] 2> 930682 T2920 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439217643359830016)} 0 10
[junit4:junit4] 2> C424_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:51113_, base_url=http://127.0.0.1:51113}
[junit4:junit4] 2> 930685 T2932 C424 P51113 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 930686 T2932 C424 P51113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 930688 T2932 C424 P51113 oass.SolrIndexSearcher.<init> Opening Searcher@251f447b realtime
[junit4:junit4] 2> 930689 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439217643359830016&update.from=http://127.0.0.1:25772/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439217643359830016)} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C428 name=collection1 org.apache.solr.core.SolrCore@420a2bf1 url=http://127.0.0.1:25772/collection1 node=127.0.0.1:25772_ C428_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:25772_, base_url=http://127.0.0.1:25772, leader=true}
[junit4:junit4] 2> 930689 T2904 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439217643359830016)} 0 17
[junit4:junit4] 2> 930690 T2949 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 21
[junit4:junit4] 2> 930691 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930693 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930695 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930697 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930698 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930701 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 930702 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930706 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930730 T2880 C427 P25060 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_2,generation=2}
[junit4:junit4] 2> 930731 T2880 C427 P25060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 930738 T2880 C427 P25060 oass.SolrIndexSearcher.<init> Opening Searcher@2f0d9e57 main
[junit4:junit4] 2> 930739 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930740 T2894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f0d9e57 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 930740 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439217643394433024)],commit=} 0 35
[junit4:junit4] 2> 930748 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25772/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439217643434278912)]} 0 1
[junit4:junit4] 2> 930749 T2905 C428 P25772 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:25772/collection1/, StdNode: http://127.0.0.1:51113/collection1/, StdNode: http://127.0.0.1:64066/collection1/, StdNode: http://127.0.0.1:35935/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 930751 T2906 C428 P25772 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930752 T2952 C425 P35935 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930751 T2922 C426 P64066 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930752 T2952 C425 P35935 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930753 T2932 C424 P51113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930753 T2922 C426 P64066 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930754 T2952 C425 P35935 oass.SolrIndexSearcher.<init> Opening Searcher@57e1dbe8 main
[junit4:junit4] 2> 930755 T2922 C426 P64066 oass.SolrIndexSearcher.<init> Opening Searcher@84e6e main
[junit4:junit4] 2> 930755 T2952 C425 P35935 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930756 T2906 C428 P25772 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 930756 T2962 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57e1dbe8 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930756 T2922 C426 P64066 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930757 T2952 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 930757 T2906 C428 P25772 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 930757 T2928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@84e6e main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930758 T2932 C424 P51113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_2,generation=2}
[junit4:junit4] 2> 930759 T2922 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 930759 T2932 C424 P51113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 930760 T2906 C428 P25772 oass.SolrIndexSearcher.<init> Opening Searcher@315049ba main
[junit4:junit4] 2> 930761 T2906 C428 P25772 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930762 T2912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@315049ba main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 930762 T2906 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 930762 T2932 C424 P51113 oass.SolrIndexSearcher.<init> Opening Searcher@71a1f660 main
[junit4:junit4] 2> 930763 T2932 C424 P51113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930764 T2944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71a1f660 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 930765 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 930765 T2905 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439217643434278912)],commit=} 0 22
[junit4:junit4] 2> 930766 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930768 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930772 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930775 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 930777 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930780 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930783 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930784 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930786 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930793 T2934 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902308&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930793 T2916 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902308&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 930796 T2934 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902308&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930797 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 930803 T2918 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902319&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 930803 T2935 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902319&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930806 T2935 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902319&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930807 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 930808 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930815 T2936 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902331&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930815 T2979 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902331&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930818 T2936 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902331&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930819 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 930825 T2937 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902341&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930825 T2982 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902341&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930828 T2937 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902341&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930829 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 930834 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930839 T2880 C427 P25060 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_3,generation=3}
[junit4:junit4] 2> 930839 T2880 C427 P25060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 930842 T2880 C427 P25060 oass.SolrIndexSearcher.<init> Opening Searcher@72f44d9 main
[junit4:junit4] 2> 930842 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930843 T2894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72f44d9 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 930844 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439217643527602176)],commit=} 0 12
[junit4:junit4] 2> 930856 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25772/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439217643545427968)]} 0 3
[junit4:junit4] 2> 930856 T2900 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:64066/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439217643545427968)]} 0 8
[junit4:junit4] 2> 930857 T2921 C426 P64066 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:25772/collection1/, StdNode: http://127.0.0.1:51113/collection1/, StdNode: http://127.0.0.1:64066/collection1/, StdNode: http://127.0.0.1:35935/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 930859 T2938 C424 P51113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930860 T2986 C426 P64066 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930861 T2900 C428 P25772 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930860 T2950 C425 P35935 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930861 T2986 C426 P64066 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930864 T2950 C425 P35935 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930864 T2986 C426 P64066 oass.SolrIndexSearcher.<init> Opening Searcher@3920c32f main
[junit4:junit4] 2> 930865 T2950 C425 P35935 oass.SolrIndexSearcher.<init> Opening Searcher@3a4a6c5c main
[junit4:junit4] 2> 930865 T2986 C426 P64066 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930866 T2928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3920c32f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930866 T2950 C425 P35935 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930867 T2986 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 930867 T2962 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a4a6c5c main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930866 T2938 C424 P51113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_3,generation=3}
[junit4:junit4] 2> 930868 T2950 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 930868 T2900 C428 P25772 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_3,generation=3}
[junit4:junit4] 2> 930869 T2938 C424 P51113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 930870 T2900 C428 P25772 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 930872 T2938 C424 P51113 oass.SolrIndexSearcher.<init> Opening Searcher@4d42d4a5 main
[junit4:junit4] 2> 930873 T2900 C428 P25772 oass.SolrIndexSearcher.<init> Opening Searcher@78a2a274 main
[junit4:junit4] 2> 930873 T2938 C424 P51113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930873 T2944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d42d4a5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 930873 T2900 C428 P25772 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930874 T2938 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 930874 T2912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78a2a274 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 930875 T2900 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 930876 T2921 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 930877 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930879 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930883 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930887 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 930888 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930891 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930894 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930895 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930897 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930903 T2916 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902419&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930903 T2934 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902419&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 930907 T2934 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902419&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930908 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 930914 T2918 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902430&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930914 T2935 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902430&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 930918 T2935 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902430&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930919 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 930920 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930927 T2979 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902443&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930927 T2936 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902443&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 930931 T2936 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902443&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930932 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 930938 T2982 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902454&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 930938 T2937 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902454&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 930942 T2937 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902454&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 930943 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 930948 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930953 T2880 C427 P25060 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_4,generation=4}
[junit4:junit4] 2> 930953 T2880 C427 P25060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 930956 T2880 C427 P25060 oass.SolrIndexSearcher.<init> Opening Searcher@69da8ab6 main
[junit4:junit4] 2> 930956 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930957 T2894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69da8ab6 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 930958 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439217643647139840)],commit=} 0 12
[junit4:junit4] 2> 930971 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25772/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439217643667062784)]} 0 2
[junit4:junit4] 2> 930972 T2902 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51113/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439217643667062784)]} 0 8
[junit4:junit4] 2> 930973 T2933 C424 P51113 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:25772/collection1/, StdNode: http://127.0.0.1:51113/collection1/, StdNode: http://127.0.0.1:64066/collection1/, StdNode: http://127.0.0.1:35935/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 930976 T2902 C428 P25772 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930977 T2992 C424 P51113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930977 T2991 C426 P64066 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930976 T2953 C425 P35935 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 930978 T2991 C426 P64066 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930979 T2953 C425 P35935 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 930980 T2991 C426 P64066 oass.SolrIndexSearcher.<init> Opening Searcher@6932be9f main
[junit4:junit4] 2> 930980 T2953 C425 P35935 oass.SolrIndexSearcher.<init> Opening Searcher@7baa09dd main
[junit4:junit4] 2> 930981 T2991 C426 P64066 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930981 T2953 C425 P35935 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930982 T2902 C428 P25772 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_4,generation=4}
[junit4:junit4] 2> 930982 T2962 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7baa09dd main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930982 T2928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6932be9f main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 930984 T2953 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 930984 T2991 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 930983 T2902 C428 P25772 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 930982 T2992 C424 P51113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_4,generation=4}
[junit4:junit4] 2> 930986 T2992 C424 P51113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 930987 T2902 C428 P25772 oass.SolrIndexSearcher.<init> Opening Searcher@33475af5 main
[junit4:junit4] 2> 930988 T2902 C428 P25772 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930988 T2992 C424 P51113 oass.SolrIndexSearcher.<init> Opening Searcher@1daf6cfc main
[junit4:junit4] 2> 930988 T2912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33475af5 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 930989 T2992 C424 P51113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 930989 T2902 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 930990 T2944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1daf6cfc main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 930991 T2992 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 930991 T2933 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 30
[junit4:junit4] 2> 930992 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930994 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 930998 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 931002 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 931003 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931006 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931009 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931010 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931012 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931017 T2993 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902534&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931018 T2916 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902534&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931022 T2993 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902534&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931023 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 931028 T2994 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902545&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931029 T2918 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902545&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931032 T2994 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902545&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931033 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 931034 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931040 T2995 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902557&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931041 T2979 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902557&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931044 T2995 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902557&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931045 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 931050 T2996 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902567&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931051 T2982 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902567&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931054 T2996 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902567&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931055 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 931060 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 931065 T2880 C427 P25060 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e7b8f8e lockFactory=org.apache.lucene.store.NativeFSLockFactory@41165746),segFN=segments_5,generation=5}
[junit4:junit4] 2> 931065 T2880 C427 P25060 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 931068 T2880 C427 P25060 oass.SolrIndexSearcher.<init> Opening Searcher@67486f93 main
[junit4:junit4] 2> 931068 T2880 C427 P25060 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 931069 T2894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67486f93 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 931070 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439217643764580352)],commit=} 0 12
[junit4:junit4] 2> 931081 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25772/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439217643782406144)]} 0 2
[junit4:junit4] 2> 931082 T2904 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35935/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439217643782406144)]} 0 7
[junit4:junit4] 2> 931082 T2951 C425 P35935 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:25772/collection1/, StdNode: http://127.0.0.1:51113/collection1/, StdNode: http://127.0.0.1:64066/collection1/, StdNode: http://127.0.0.1:35935/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 931084 T2999 C424 P51113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 931084 T2954 C425 P35935 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 931085 T2920 C426 P64066 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 931085 T2904 C428 P25772 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 931086 T2920 C426 P64066 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 931085 T2954 C425 P35935 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 931087 T2920 C426 P64066 oass.SolrIndexSearcher.<init> Opening Searcher@2f9ee270 main
[junit4:junit4] 2> 931087 T2954 C425 P35935 oass.SolrIndexSearcher.<init> Opening Searcher@241e913c main
[junit4:junit4] 2> 931088 T2920 C426 P64066 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 931089 T2928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f9ee270 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 931088 T2954 C425 P35935 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 931089 T2920 C426 P64066 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 931089 T2962 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@241e913c main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 931089 T2999 C424 P51113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fca61e0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b74de8d),segFN=segments_5,generation=5}
[junit4:junit4] 2> 931091 T2954 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 931090 T2904 C428 P25772 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3379513f lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f5a0c4a),segFN=segments_5,generation=5}
[junit4:junit4] 2> 931091 T2999 C424 P51113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 931092 T2904 C428 P25772 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 931094 T2999 C424 P51113 oass.SolrIndexSearcher.<init> Opening Searcher@7dc833b main
[junit4:junit4] 2> 931095 T2904 C428 P25772 oass.SolrIndexSearcher.<init> Opening Searcher@73264524 main
[junit4:junit4] 2> 931095 T2999 C424 P51113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 931095 T2944 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7dc833b main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 931095 T2904 C428 P25772 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 931096 T2999 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 931096 T2912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73264524 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 931097 T2904 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 931098 T2951 C425 P35935 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 931099 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931101 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931105 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 931109 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931110 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931113 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931116 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931117 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931118 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931123 T2955 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902640&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931124 T2993 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902640&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931127 T2993 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902640&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931128 T2905 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 931133 T3000 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902650&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931134 T2994 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902650&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931137 T2994 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902650&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931138 T2933 C424 P51113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 931139 T2868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 931145 T3001 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902662&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931146 T2995 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902662&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931150 T2995 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902662&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931151 T2921 C426 P64066 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 931156 T3002 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902673&shard.url=127.0.0.1:64066/collection1/|127.0.0.1:35935/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 931157 T2996 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902673&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 931161 T2996 C428 P25772 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372544902673&shard.url=127.0.0.1:25772/collection1/|127.0.0.1:51113/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 931162 T2951 C425 P35935 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 931170 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439217643877826560)]} 0 4
[junit4:junit4] 2> 931185 T2932 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:25772/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439217643887263744)]} 0 4
[junit4:junit4] 2> 931186 T2902 C428 P25772 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51113/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439217643887263744)]} 0 11
[junit4:junit4] 2> 931187 T3003 C424 P51113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 15
[junit4:junit4] 2> 931192 T2880 C427 P25060 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt
[...truncated too long message...]
ngsFormat(name=Lucene41WithOrds), name=PostingsFormat(name=Lucene41WithOrds), foo_i=Lucene41(blocksize=128), regex_dup_B_s=PostingsFormat(name=TestBloomFilteredLucene41Postings), multiDefault=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tf1=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_dt1=Lucene41(blocksize=128), genre_s=PostingsFormat(name=Lucene41WithOrds), author_t=Lucene41(blocksize=128), range_facet_l=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_ti1=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=NestedPulsing), _version_=Lucene41(blocksize=128), val_i=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=Lucene41(blocksize=128), series_t=Lucene41(blocksize=128), a_t=PostingsFormat(name=TestBloomFilteredLucene41Postings), n_tdt1=Lucene41(blocksize=128), regex_dup_A_s=PostingsFormat(name=NestedPulsing), price=PostingsFormat(name=NestedPulsing), other_tl1=Lucene41(blocksize=128), n_l1=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_si=Lucene41(blocksize=128), inStock=PostingsFormat(name=NestedPulsing)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=ar_LY, timezone=Asia/Muscat
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=331147856,total=507510784
[junit4:junit4] 2> NOTE: All tests run in this JVM: [RecoveryZkTest, TestDefaultSearchFieldResource, UpdateRequestProcessorFactoryTest, TestManagedSchemaFieldResource, TestSort, DocValuesTest, TestFieldTypeResource, ShowFileRequestHandlerTest, TestIBSimilarityFactory, TestCharFilters, TestDocSet, PreAnalyzedFieldTest, UnloadDistributedZkTest, TestPropInjectDefaults, SearchHandlerTest, CoreAdminHandlerTest, TestStressVersions, FastVectorHighlighterTest, TestReversedWildcardFilterFactory, DistributedQueryElevationComponentTest, DocumentBuilderTest, DateFieldTest, FieldMutatingUpdateProcessorTest, CacheHeaderTest, MinimalSchemaTest, SpellingQueryConverterTest, TestSweetSpotSimilarityFactory, TestMaxScoreQueryParser, AutoCommitTest, ParsingFieldUpdateProcessorsTest, TestFuzzyAnalyzedSuggestions, BasicFunctionalityTest, TermVectorComponentTest, ZkCLITest, BadCopyFieldTest, TestDynamicFieldCollectionResource, TestShardHandlerFactory, QueryElevationComponentTest, TermsComponentTest, ScriptEngineTest, TestSolrXmlPersistence, AnalysisAfterCoreReloadTest, IndexSchemaTest, AliasIntegrationTest, DistributedSpellCheckComponentTest, ResponseLogComponentTest, TestReplicationHandler, DirectUpdateHandlerOptimizeTest, TestSurroundQueryParser, TestLRUCache, TestFieldTypeCollectionResource, TestCoreContainer, BlockDirectoryTest, TestStressRecovery, SolrCoreTest, SliceStateUpdateTest, OverseerTest, TestQueryUtils, MBeansHandlerTest, PrimitiveFieldTypeTest, TestSerializedLuceneMatchVersion, TestFastLRUCache, TestSolrCoreProperties, TestPhraseSuggestions, TestBadConfig, SolrInfoMBeanTest, TestLMDirichletSimilarityFactory, ClusterStateUpdateTest, BinaryUpdateRequestHandlerTest, TestLFUCache, DirectSolrConnectionTest, OverseerCollectionProcessorTest, SyncSliceTest, RegexBoostProcessorTest, QueryParsingTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=91C3E9011B6A65D8 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_LY -Dtests.timezone=Asia/Muscat -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([91C3E9011B6A65D8]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=3325, name=recoveryCmdExecutor-1563-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([91C3E9011B6A65D8]:0)
[junit4:junit4] Completed on J0 in 237.12s, 1 test, 2 errors <<< FAILURES!
[...truncated 519 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 584 ignored (6 assumptions)
Total time: 44 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4098 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4098/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([3F36C5300C3BE0C4]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([3F36C5300C3BE0C4]:0)
Build Log:
[...truncated 9471 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 617348 T1265 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 617353 T1265 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372510639357
[junit4:junit4] 2> 617354 T1265 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 617355 T1266 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 617457 T1265 oasc.ZkTestServer.run start zk server on port:25988
[junit4:junit4] 2> 617458 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 617462 T1272 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36ff2e3c name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 617463 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 617463 T1265 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 617475 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 617477 T1274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ae27547 name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 617477 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 617477 T1265 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 617488 T1265 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 617491 T1265 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 617494 T1265 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 617497 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 617498 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 617510 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 617511 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 617616 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 617617 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 617620 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 617621 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 617631 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 617631 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 617635 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 617635 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 617639 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 617639 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 617642 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 617643 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 617646 T1265 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 617647 T1265 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 617922 T1265 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 617927 T1265 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17142
[junit4:junit4] 2> 617928 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 617928 T1265 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 617929 T1265 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661
[junit4:junit4] 2> 617929 T1265 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/solr.xml
[junit4:junit4] 2> 617930 T1265 oasc.CoreContainer.<init> New CoreContainer 1567207044
[junit4:junit4] 2> 617930 T1265 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/'
[junit4:junit4] 2> 617931 T1265 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/'
[junit4:junit4] 2> 618063 T1265 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 618063 T1265 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 618064 T1265 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 618064 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 618065 T1265 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 618065 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 618066 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 618066 T1265 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 618067 T1265 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 618067 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 618086 T1265 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 618087 T1265 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:25988/solr
[junit4:junit4] 2> 618087 T1265 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 618089 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 618091 T1285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@441c3e18 name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 618092 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 618104 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 618118 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 618120 T1287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41f67a4a name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 618120 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 618123 T1265 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 618132 T1265 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 618136 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 618138 T1265 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17142_
[junit4:junit4] 2> 618146 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17142_
[junit4:junit4] 2> 618149 T1265 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 618163 T1265 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 618170 T1265 oasc.Overseer.start Overseer (id=89948857261359107-127.0.0.1:17142_-n_0000000000) starting
[junit4:junit4] 2> 618176 T1265 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 618187 T1289 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 618188 T1265 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 618191 T1265 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 618198 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 618200 T1288 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 618207 T1290 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 618208 T1290 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 618239 T1290 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 619706 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 619707 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:17142_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:17142"}
[junit4:junit4] 2> 619707 T1288 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 619707 T1288 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 619728 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 620243 T1290 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 620243 T1290 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/collection1
[junit4:junit4] 2> 620243 T1290 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 620244 T1290 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 620244 T1290 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 620246 T1290 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/collection1/'
[junit4:junit4] 2> 620248 T1290 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/collection1/lib/README' to classloader
[junit4:junit4] 2> 620248 T1290 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 620302 T1290 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 620363 T1290 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 620466 T1290 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 620474 T1290 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 620880 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=436281678
[junit4:junit4] 2> 621123 T1290 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 621137 T1290 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 621141 T1290 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 621157 T1290 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 621162 T1290 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 621167 T1290 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 621168 T1290 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 621169 T1290 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 621169 T1290 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 621170 T1290 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 621171 T1290 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 621171 T1290 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 621171 T1290 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372510639661/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/control/data/
[junit4:junit4] 2> 621172 T1290 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e356b5b
[junit4:junit4] 2> 621173 T1290 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/control/data
[junit4:junit4] 2> 621173 T1290 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/control/data/index/
[junit4:junit4] 2> 621173 T1290 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 621174 T1290 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/control/data/index
[junit4:junit4] 2> 621178 T1290 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_1,generation=1}
[junit4:junit4] 2> 621179 T1290 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 621182 T1290 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 621182 T1290 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 621183 T1290 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 621183 T1290 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 621184 T1290 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 621184 T1290 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 621184 T1290 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 621185 T1290 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 621185 T1290 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 621202 T1290 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 621209 T1290 oass.SolrIndexSearcher.<init> Opening Searcher@439fa283 main
[junit4:junit4] 2> 621210 T1290 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 621211 T1290 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 621215 T1291 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@439fa283 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 621217 T1290 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 621217 T1290 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:17142 collection:control_collection shard:shard1
[junit4:junit4] 2> 621218 T1290 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 621236 T1290 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 621246 T1290 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 621246 T1290 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 621246 T1290 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:17142/collection1/
[junit4:junit4] 2> 621247 T1290 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 621247 T1290 oasc.SyncStrategy.syncToMe http://127.0.0.1:17142/collection1/ has no replicas
[junit4:junit4] 2> 621247 T1290 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:17142/collection1/ shard1
[junit4:junit4] 2> 621247 T1290 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 622739 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 622756 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 622807 T1290 oasc.ZkController.register We are http://127.0.0.1:17142/collection1/ and leader is http://127.0.0.1:17142/collection1/
[junit4:junit4] 2> 622807 T1290 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:17142
[junit4:junit4] 2> 622807 T1290 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 622807 T1290 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 622808 T1290 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 622810 T1290 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 622811 T1265 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 622812 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 622812 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 622831 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 622833 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 622835 T1294 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34827401 name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 622835 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 622837 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 622839 T1265 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 623108 T1265 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 623111 T1265 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15050
[junit4:junit4] 2> 623112 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 623112 T1265 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 623113 T1265 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843
[junit4:junit4] 2> 623113 T1265 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/solr.xml
[junit4:junit4] 2> 623114 T1265 oasc.CoreContainer.<init> New CoreContainer 2017123388
[junit4:junit4] 2> 623114 T1265 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/'
[junit4:junit4] 2> 623115 T1265 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/'
[junit4:junit4] 2> 623249 T1265 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 623250 T1265 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 623250 T1265 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 623251 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 623251 T1265 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 623252 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 623252 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 623253 T1265 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 623253 T1265 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 623254 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 623271 T1265 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 623272 T1265 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:25988/solr
[junit4:junit4] 2> 623272 T1265 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 623273 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 623326 T1305 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4838a78a name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 623326 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 623329 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 623341 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 623343 T1307 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab37e6a name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 623343 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 623381 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 624271 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 624272 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:17142_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:17142"}
[junit4:junit4] 2> 624302 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 624302 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 624302 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 624386 T1265 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15050_
[junit4:junit4] 2> 624388 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15050_
[junit4:junit4] 2> 624390 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 624390 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 624391 T1294 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 624391 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 624392 T1287 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 624399 T1307 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 624405 T1308 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 624405 T1308 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 624407 T1308 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 625815 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 625816 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:15050_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:15050"}
[junit4:junit4] 2> 625816 T1288 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 625816 T1288 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 625832 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 625832 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 625832 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 626411 T1308 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 626411 T1308 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/collection1
[junit4:junit4] 2> 626411 T1308 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 626412 T1308 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 626412 T1308 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 626414 T1308 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/collection1/'
[junit4:junit4] 2> 626416 T1308 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 626416 T1308 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/collection1/lib/README' to classloader
[junit4:junit4] 2> 626469 T1308 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 626531 T1308 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 626633 T1308 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 626642 T1308 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 627279 T1308 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 627294 T1308 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 627299 T1308 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 627315 T1308 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 627320 T1308 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 627325 T1308 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 627326 T1308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 627327 T1308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 627327 T1308 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 627328 T1308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 627328 T1308 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 627329 T1308 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 627329 T1308 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372510644843/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty1/
[junit4:junit4] 2> 627329 T1308 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e356b5b
[junit4:junit4] 2> 627330 T1308 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty1
[junit4:junit4] 2> 627331 T1308 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty1/index/
[junit4:junit4] 2> 627331 T1308 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 627331 T1308 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty1/index
[junit4:junit4] 2> 627335 T1308 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> 627335 T1308 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 627339 T1308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 627339 T1308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 627340 T1308 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 627340 T1308 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 627341 T1308 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 627341 T1308 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 627342 T1308 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 627342 T1308 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 627343 T1308 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 627359 T1308 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 627367 T1308 oass.SolrIndexSearcher.<init> Opening Searcher@64b4b4a0 main
[junit4:junit4] 2> 627368 T1308 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 627368 T1308 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 627374 T1309 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64b4b4a0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 627376 T1308 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 627376 T1308 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15050 collection:collection1 shard:shard1
[junit4:junit4] 2> 627377 T1308 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 627403 T1308 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 627412 T1308 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 627413 T1308 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 627413 T1308 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15050/collection1/
[junit4:junit4] 2> 627413 T1308 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 627413 T1308 oasc.SyncStrategy.syncToMe http://127.0.0.1:15050/collection1/ has no replicas
[junit4:junit4] 2> 627414 T1308 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15050/collection1/ shard1
[junit4:junit4] 2> 627414 T1308 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 628839 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 628864 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 628864 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 628864 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 628919 T1308 oasc.ZkController.register We are http://127.0.0.1:15050/collection1/ and leader is http://127.0.0.1:15050/collection1/
[junit4:junit4] 2> 628919 T1308 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15050
[junit4:junit4] 2> 628919 T1308 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 628919 T1308 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 628920 T1308 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 628922 T1308 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 628923 T1265 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 628924 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 628924 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 629213 T1265 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 629217 T1265 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22580
[junit4:junit4] 2> 629217 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 629218 T1265 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 629218 T1265 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944
[junit4:junit4] 2> 629219 T1265 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/solr.xml
[junit4:junit4] 2> 629219 T1265 oasc.CoreContainer.<init> New CoreContainer 1027282630
[junit4:junit4] 2> 629220 T1265 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/'
[junit4:junit4] 2> 629221 T1265 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/'
[junit4:junit4] 2> 629353 T1265 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 629354 T1265 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 629355 T1265 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 629355 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 629356 T1265 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 629356 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 629356 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 629357 T1265 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 629357 T1265 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 629358 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 629375 T1265 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 629375 T1265 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:25988/solr
[junit4:junit4] 2> 629376 T1265 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 629377 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 629390 T1321 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d0da09f name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 629390 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 629408 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 629420 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 629422 T1323 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@108cd34a name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 629422 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 629435 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 630371 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 630372 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:15050_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:15050"}
[junit4:junit4] 2> 630386 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630386 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630386 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630386 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630439 T1265 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22580_
[junit4:junit4] 2> 630441 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22580_
[junit4:junit4] 2> 630444 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630444 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630444 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 630444 T1294 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 630444 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 630445 T1287 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 630446 T1307 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 630452 T1323 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 630457 T1324 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 630458 T1324 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 630459 T1324 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 631892 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 631893 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:22580_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:22580"}
[junit4:junit4] 2> 631893 T1288 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 631893 T1288 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 631909 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 631909 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 631909 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 631909 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 632463 T1324 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 632463 T1324 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/collection1
[junit4:junit4] 2> 632463 T1324 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 632464 T1324 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 632464 T1324 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 632466 T1324 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/collection1/'
[junit4:junit4] 2> 632468 T1324 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 632468 T1324 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/collection1/lib/README' to classloader
[junit4:junit4] 2> 632521 T1324 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 632583 T1324 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 632685 T1324 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 632693 T1324 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 633309 T1324 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 633323 T1324 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 633327 T1324 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 633344 T1324 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 633349 T1324 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 633354 T1324 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 633356 T1324 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 633356 T1324 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 633357 T1324 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 633358 T1324 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 633358 T1324 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 633358 T1324 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 633359 T1324 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372510650944/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty2/
[junit4:junit4] 2> 633359 T1324 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e356b5b
[junit4:junit4] 2> 633360 T1324 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty2
[junit4:junit4] 2> 633361 T1324 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty2/index/
[junit4:junit4] 2> 633361 T1324 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 633361 T1324 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty2/index
[junit4:junit4] 2> 633365 T1324 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2e4aa6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5e31c91f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 633365 T1324 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 633368 T1324 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 633369 T1324 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 633369 T1324 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 633370 T1324 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 633371 T1324 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 633371 T1324 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 633371 T1324 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 633372 T1324 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 633372 T1324 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 633389 T1324 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 633396 T1324 oass.SolrIndexSearcher.<init> Opening Searcher@5a90ec97 main
[junit4:junit4] 2> 633398 T1324 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 633398 T1324 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 633403 T1325 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a90ec97 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 633405 T1324 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 633405 T1324 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22580 collection:collection1 shard:shard2
[junit4:junit4] 2> 633406 T1324 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 633472 T1324 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 633502 T1324 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 633503 T1324 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 633503 T1324 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22580/collection1/
[junit4:junit4] 2> 633503 T1324 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 633503 T1324 oasc.SyncStrategy.syncToMe http://127.0.0.1:22580/collection1/ has no replicas
[junit4:junit4] 2> 633503 T1324 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22580/collection1/ shard2
[junit4:junit4] 2> 633504 T1324 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 634976 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 635000 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 635000 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 635001 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 635001 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 635059 T1324 oasc.ZkController.register We are http://127.0.0.1:22580/collection1/ and leader is http://127.0.0.1:22580/collection1/
[junit4:junit4] 2> 635059 T1324 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22580
[junit4:junit4] 2> 635059 T1324 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 635059 T1324 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 635060 T1324 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 635062 T1324 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 635063 T1265 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 635064 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 635064 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 635350 T1265 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 635354 T1265 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18829
[junit4:junit4] 2> 635354 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 635355 T1265 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 635355 T1265 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086
[junit4:junit4] 2> 635356 T1265 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/solr.xml
[junit4:junit4] 2> 635356 T1265 oasc.CoreContainer.<init> New CoreContainer 2001874191
[junit4:junit4] 2> 635357 T1265 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/'
[junit4:junit4] 2> 635357 T1265 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/'
[junit4:junit4] 2> 635489 T1265 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 635490 T1265 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 635490 T1265 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 635491 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 635491 T1265 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 635492 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 635492 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 635493 T1265 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 635493 T1265 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 635494 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 635510 T1265 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 635511 T1265 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:25988/solr
[junit4:junit4] 2> 635511 T1265 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 635512 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 635516 T1337 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@147f4d30 name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 635516 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 635531 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 635543 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 635545 T1339 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a75b53b name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 635545 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 635557 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 636516 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 636517 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:22580_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:22580"}
[junit4:junit4] 2> 636532 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636532 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636532 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636532 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636532 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636562 T1265 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18829_
[junit4:junit4] 2> 636565 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18829_
[junit4:junit4] 2> 636568 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636568 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636568 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636568 T1294 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636568 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 636569 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 636570 T1287 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636570 T1323 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636571 T1307 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636571 T1339 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 636576 T1340 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 636577 T1340 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 636578 T1340 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 638039 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 638040 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:18829_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:18829"}
[junit4:junit4] 2> 638040 T1288 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 638040 T1288 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 638055 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 638055 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 638055 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 638055 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 638055 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 638582 T1340 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 638582 T1340 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/collection1
[junit4:junit4] 2> 638582 T1340 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 638583 T1340 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 638584 T1340 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 638585 T1340 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/collection1/'
[junit4:junit4] 2> 638587 T1340 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 638587 T1340 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/collection1/lib/README' to classloader
[junit4:junit4] 2> 638658 T1340 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 638720 T1340 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 638822 T1340 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 638830 T1340 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 639422 T1340 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 639431 T1340 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 639434 T1340 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 639448 T1340 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 639452 T1340 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 639455 T1340 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 639456 T1340 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 639457 T1340 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 639457 T1340 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 639458 T1340 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 639458 T1340 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 639458 T1340 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 639459 T1340 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372510657086/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty3/
[junit4:junit4] 2> 639459 T1340 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e356b5b
[junit4:junit4] 2> 639460 T1340 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty3
[junit4:junit4] 2> 639460 T1340 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty3/index/
[junit4:junit4] 2> 639460 T1340 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 639460 T1340 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty3/index
[junit4:junit4] 2> 639463 T1340 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 639463 T1340 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 639465 T1340 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 639465 T1340 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 639466 T1340 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 639466 T1340 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 639467 T1340 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 639467 T1340 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 639467 T1340 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 639468 T1340 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 639468 T1340 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 639479 T1340 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 639483 T1340 oass.SolrIndexSearcher.<init> Opening Searcher@723ea6c2 main
[junit4:junit4] 2> 639484 T1340 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 639484 T1340 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 639488 T1341 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@723ea6c2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 639490 T1340 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 639490 T1340 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18829 collection:collection1 shard:shard1
[junit4:junit4] 2> 639498 T1340 oasc.ZkController.register We are http://127.0.0.1:18829/collection1/ and leader is http://127.0.0.1:15050/collection1/
[junit4:junit4] 2> 639498 T1340 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18829
[junit4:junit4] 2> 639498 T1340 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 639498 T1340 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C387 name=collection1 org.apache.solr.core.SolrCore@1cc0dce3 url=http://127.0.0.1:18829/collection1 node=127.0.0.1:18829_ C387_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:18829_, base_url=http://127.0.0.1:18829}
[junit4:junit4] 2> 639499 T1342 C387 P18829 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 639499 T1340 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 639499 T1342 C387 P18829 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 639499 T1342 C387 P18829 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 639500 T1342 C387 P18829 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 639500 T1265 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 639500 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 639501 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 639501 T1342 C387 P18829 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 639519 T1297 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 639560 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 639561 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:18829_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:18829"}
[junit4:junit4] 2> 639577 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639577 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639577 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639577 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639577 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 639693 T1265 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 639695 T1265 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:37001
[junit4:junit4] 2> 639696 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 639696 T1265 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 639696 T1265 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520
[junit4:junit4] 2> 639697 T1265 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/solr.xml
[junit4:junit4] 2> 639697 T1265 oasc.CoreContainer.<init> New CoreContainer 555348712
[junit4:junit4] 2> 639697 T1265 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/'
[junit4:junit4] 2> 639698 T1265 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/'
[junit4:junit4] 2> 639784 T1265 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 639785 T1265 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 639785 T1265 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 639786 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 639786 T1265 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 639786 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 639787 T1265 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 639787 T1265 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 639787 T1265 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 639788 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 639799 T1265 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 639800 T1265 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:25988/solr
[junit4:junit4] 2> 639800 T1265 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 639801 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 639804 T1354 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78f22f2c name:ZooKeeperConnection Watcher:127.0.0.1:25988 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 639804 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 639818 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 639831 T1265 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 639833 T1356 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@764da833 name:ZooKeeperConnection Watcher:127.0.0.1:25988/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 639833 T1265 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 639845 T1265 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 640520 T1297 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 640520 T1297 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:18829_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 640850 T1265 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37001_
[junit4:junit4] 2> 640860 T1265 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37001_
[junit4:junit4] 2> 640863 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 640863 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 640863 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 640863 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 640864 T1294 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640864 T1356 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640865 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 640866 T1287 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640866 T1323 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640866 T1307 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640866 T1339 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 640882 T1357 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 640882 T1357 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 640884 T1357 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 641082 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 641083 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37001_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37001"}
[junit4:junit4] 2> 641083 T1288 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 641084 T1288 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 641097 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641098 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641098 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641098 T1356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641098 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641097 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 641885 T1357 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 641885 T1357 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/collection1
[junit4:junit4] 2> 641885 T1357 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 641886 T1357 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 641886 T1357 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 641888 T1357 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/collection1/'
[junit4:junit4] 2> 641889 T1357 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/collection1/lib/README' to classloader
[junit4:junit4] 2> 641890 T1357 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 641942 T1357 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 642004 T1357 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 642106 T1357 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 642113 T1357 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C388 name=collection1 org.apache.solr.core.SolrCore@1cc0dce3 url=http://127.0.0.1:18829/collection1 node=127.0.0.1:18829_ C388_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:18829_, base_url=http://127.0.0.1:18829}
[junit4:junit4] 2> 642522 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:15050/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 642522 T1342 C388 P18829 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:18829 START replicas=[http://127.0.0.1:15050/collection1/] nUpdates=100
[junit4:junit4] 2> 642523 T1342 C388 P18829 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 642523 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 642524 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 642524 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 642524 T1342 C388 P18829 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 642524 T1342 C388 P18829 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:15050/collection1/. core=collection1
[junit4:junit4] 2> 642525 T1342 C388 P18829 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C389 name=collection1 org.apache.solr.core.SolrCore@63069b01 url=http://127.0.0.1:15050/collection1 node=127.0.0.1:15050_ C389_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:15050_, base_url=http://127.0.0.1:15050, leader=true}
[junit4:junit4] 2> 642532 T1298 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 642538 T1299 C389 P15050 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 642541 T1299 C389 P15050 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> 642542 T1299 C389 P15050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 642542 T1299 C389 P15050 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 642543 T1299 C389 P15050 oass.SolrIndexSearcher.<init> Opening Searcher@462b709e realtime
[junit4:junit4] 2> 642543 T1299 C389 P15050 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 642543 T1299 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 642544 T1342 C388 P18829 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 642545 T1342 C388 P18829 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 642547 T1300 C389 P15050 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 642547 T1300 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 642548 T1342 C388 P18829 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 642548 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 642548 T1342 C388 P18829 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 642548 T1342 C388 P18829 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 642550 T1342 C388 P18829 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 642605 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 642605 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:18829_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:18829"}
[junit4:junit4] 2> 642619 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642619 T1356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642619 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642619 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642619 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642619 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 642745 T1357 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 642759 T1357 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 642763 T1357 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 642780 T1357 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 642785 T1357 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 642790 T1357 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 642791 T1357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 642791 T1357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 642792 T1357 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 642793 T1357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 642793 T1357 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 642793 T1357 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 642794 T1357 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372510661520/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty4/
[junit4:junit4] 2> 642794 T1357 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e356b5b
[junit4:junit4] 2> 642795 T1357 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty4
[junit4:junit4] 2> 642795 T1357 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty4/index/
[junit4:junit4] 2> 642796 T1357 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 642796 T1357 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372510639357/jetty4/index
[junit4:junit4] 2> 642800 T1357 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@78ef6aa4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7534ae98),segFN=segments_1,generation=1}
[junit4:junit4] 2> 642800 T1357 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 642803 T1357 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 642803 T1357 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 642804 T1357 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 642805 T1357 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 642805 T1357 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 642806 T1357 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 642806 T1357 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 642807 T1357 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 642808 T1357 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 642824 T1357 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 642831 T1357 oass.SolrIndexSearcher.<init> Opening Searcher@75b2b64 main
[junit4:junit4] 2> 642833 T1357 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 642833 T1357 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 642837 T1359 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75b2b64 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 642839 T1357 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 642840 T1357 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37001 collection:collection1 shard:shard2
[junit4:junit4] 2> 642843 T1357 oasc.ZkController.register We are http://127.0.0.1:37001/collection1/ and leader is http://127.0.0.1:22580/collection1/
[junit4:junit4] 2> 642843 T1357 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37001
[junit4:junit4] 2> 642843 T1357 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 642844 T1357 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C390 name=collection1 org.apache.solr.core.SolrCore@21a1a43d url=http://127.0.0.1:37001/collection1 node=127.0.0.1:37001_ C390_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:37001_, base_url=http://127.0.0.1:37001}
[junit4:junit4] 2> 642844 T1360 C390 P37001 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 642845 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 642845 T1357 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 642845 T1360 C390 P37001 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 642845 T1360 C390 P37001 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 642846 T1265 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 642847 T1265 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 642847 T1360 C390 P37001 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 642847 T1265 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 642867 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 642868 T1313 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 642868 T1265 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 642869 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 643872 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 644125 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 644126 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37001_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37001"}
[junit4:junit4] 2> 644146 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644146 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644146 T1356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644146 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644146 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644146 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 644871 T1313 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 644871 T1313 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:37001_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 644876 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 645879 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C390_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:37001_, base_url=http://127.0.0.1:37001}
[junit4:junit4] 2> 646873 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22580/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 646873 T1360 C390 P37001 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37001 START replicas=[http://127.0.0.1:22580/collection1/] nUpdates=100
[junit4:junit4] 2> 646873 T1360 C390 P37001 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 646874 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 646874 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 646874 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 646874 T1360 C390 P37001 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 646875 T1360 C390 P37001 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22580/collection1/. core=collection1
[junit4:junit4] 2> 646875 T1360 C390 P37001 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 646881 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C391 name=collection1 org.apache.solr.core.SolrCore@4b27522c url=http://127.0.0.1:22580/collection1 node=127.0.0.1:22580_ C391_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:22580_, base_url=http://127.0.0.1:22580, leader=true}
[junit4:junit4] 2> 646883 T1314 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 646888 T1315 C391 P22580 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 646891 T1315 C391 P22580 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2e4aa6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5e31c91f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 646892 T1315 C391 P22580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 646892 T1315 C391 P22580 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 646893 T1315 C391 P22580 oass.SolrIndexSearcher.<init> Opening Searcher@7267316d realtime
[junit4:junit4] 2> 646893 T1315 C391 P22580 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 646894 T1315 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 646895 T1360 C390 P37001 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 646895 T1360 C390 P37001 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 646897 T1316 C391 P22580 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 646897 T1316 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 646898 T1360 C390 P37001 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 646898 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 646898 T1360 C390 P37001 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 646898 T1360 C390 P37001 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 646900 T1360 C390 P37001 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 647155 T1288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647156 T1288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37001_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37001"}
[junit4:junit4] 2> 647174 T1287 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647174 T1307 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647174 T1294 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647174 T1339 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647174 T1356 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647174 T1323 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 647884 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647886 T1265 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C392 name=collection1 org.apache.solr.core.SolrCore@31862bb3 url=http://127.0.0.1:17142/collection1 node=127.0.0.1:17142_ C392_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:17142_, base_url=http://127.0.0.1:17142, leader=true}
[junit4:junit4] 2> 647898 T1277 C392 P17142 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_1,generation=1}
[junit4:junit4] 2> 647899 T1277 C392 P17142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 647900 T1277 C392 P17142 oass.SolrIndexSearcher.<init> Opening Searcher@3e1f8ec7 realtime
[junit4:junit4] 2> 647901 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439181748198965248)} 0 7
[junit4:junit4] 2> C390_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:37001_, base_url=http://127.0.0.1:37001}
[junit4:junit4] 2> 647930 T1347 C390 P37001 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@78ef6aa4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7534ae98),segFN=segments_1,generation=1}
[junit4:junit4] 2> 647931 T1347 C390 P37001 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 647932 T1347 C390 P37001 oass.SolrIndexSearcher.<init> Opening Searcher@5da19ba2 realtime
[junit4:junit4] 2> 647933 T1347 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439181748228325376&update.from=http://127.0.0.1:22580/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439181748228325376)} 0 5
[junit4:junit4] 2> 647933 T1317 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439181748228325376)} 0 10
[junit4:junit4] 2> C388_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:18829_, base_url=http://127.0.0.1:18829}
[junit4:junit4] 2> 647937 T1329 C388 P18829 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 647938 T1329 C388 P18829 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 647939 T1329 C388 P18829 oass.SolrIndexSearcher.<init> Opening Searcher@3715e838 realtime
[junit4:junit4] 2> 647940 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439181748229373952&update.from=http://127.0.0.1:15050/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439181748229373952)} 0 5
[junit4:junit4] 2> 647941 T1301 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439181748229373952)} 0 17
[junit4:junit4] 2> 647941 T1346 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 20
[junit4:junit4] 2> 647942 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647944 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647947 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 647949 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 647950 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647952 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 647954 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 647960 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 647963 T1277 C392 P17142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_2,generation=2}
[junit4:junit4] 2> 647963 T1277 C392 P17142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 647965 T1277 C392 P17142 oass.SolrIndexSearcher.<init> Opening Searcher@3855be3c main
[junit4:junit4] 2> 647965 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 647966 T1291 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3855be3c main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 647967 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439181748265025536)],commit=} 0 9
[junit4:junit4] 2> 647975 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:15050/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439181748276559872)]} 0 2
[junit4:junit4] 2> 647976 T1302 C389 P15050 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:15050/collection1/, StdNode: http://127.0.0.1:18829/collection1/, StdNode: http://127.0.0.1:22580/collection1/, StdNode: http://127.0.0.1:37001/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 647978 T1303 C389 P15050 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 647978 T1319 C391 P22580 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 647979 T1349 C390 P37001 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 647979 T1329 C388 P18829 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 647979 T1319 C391 P22580 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 647979 T1349 C390 P37001 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 647980 T1319 C391 P22580 oass.SolrIndexSearcher.<init> Opening Searcher@7b2b2782 main
[junit4:junit4] 2> 647981 T1349 C390 P37001 oass.SolrIndexSearcher.<init> Opening Searcher@69886265 main
[junit4:junit4] 2> 647981 T1319 C391 P22580 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 647981 T1303 C389 P15050 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_2,generation=2}
[junit4:junit4] 2> 647982 T1359 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69886265 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 647983 T1303 C389 P15050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 647982 T1329 C388 P18829 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 647982 T1325 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b2b2782 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 647982 T1349 C390 P37001 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 647984 T1319 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 647984 T1303 C389 P15050 oass.SolrIndexSearcher.<init> Opening Searcher@2bbfcaa4 main
[junit4:junit4] 2> 647984 T1329 C388 P18829 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 647985 T1303 C389 P15050 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 647984 T1349 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 647986 T1309 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2bbfcaa4 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 647987 T1329 C388 P18829 oass.SolrIndexSearcher.<init> Opening Searcher@13f650b8 main
[junit4:junit4] 2> 647987 T1303 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 647987 T1329 C388 P18829 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 647988 T1341 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13f650b8 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 647988 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 647989 T1302 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439181748276559872)],commit=} 0 20
[junit4:junit4] 2> 647990 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647992 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 647996 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 647999 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648001 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648005 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 648007 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648008 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648010 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648017 T1350 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670017&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648017 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670017&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648022 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670017&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648023 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648030 T1351 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670030&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 648030 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670030&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648033 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670030&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648034 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 648035 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648043 T1352 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670043&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 648043 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670043&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648047 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670043&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648048 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648054 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670054&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648054 T1380 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670054&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648058 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670054&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648059 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648065 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648068 T1277 C392 P17142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_3,generation=3}
[junit4:junit4] 2> 648068 T1277 C392 P17142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 648070 T1277 C392 P17142 oass.SolrIndexSearcher.<init> Opening Searcher@64eb4a16 main
[junit4:junit4] 2> 648070 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648071 T1291 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64eb4a16 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 648072 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439181748374077440)],commit=} 0 10
[junit4:junit4] 2> 648085 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:15050/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439181748389806080)]} 0 3
[junit4:junit4] 2> 648085 T1381 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:22580/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439181748389806080)]} 0 8
[junit4:junit4] 2> 648086 T1318 C391 P22580 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:15050/collection1/, StdNode: http://127.0.0.1:18829/collection1/, StdNode: http://127.0.0.1:22580/collection1/, StdNode: http://127.0.0.1:37001/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 648088 T1331 C388 P18829 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648088 T1313 C391 P22580 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648090 T1313 C391 P22580 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648090 T1347 C390 P37001 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648090 T1313 C391 P22580 oass.SolrIndexSearcher.<init> Opening Searcher@4fd84783 main
[junit4:junit4] 2> 648090 T1381 C389 P15050 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648092 T1313 C391 P22580 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648091 T1347 C390 P37001 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648093 T1325 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4fd84783 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648094 T1331 C388 P18829 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_3,generation=3}
[junit4:junit4] 2> 648094 T1347 C390 P37001 oass.SolrIndexSearcher.<init> Opening Searcher@59395332 main
[junit4:junit4] 2> 648094 T1331 C388 P18829 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 648094 T1313 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 648096 T1381 C389 P15050 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_3,generation=3}
[junit4:junit4] 2> 648095 T1347 C390 P37001 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648097 T1331 C388 P18829 oass.SolrIndexSearcher.<init> Opening Searcher@1ea70fa9 main
[junit4:junit4] 2> 648096 T1381 C389 P15050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 648096 T1359 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59395332 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648097 T1331 C388 P18829 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648098 T1341 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ea70fa9 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 648098 T1347 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 648099 T1381 C389 P15050 oass.SolrIndexSearcher.<init> Opening Searcher@3a3cb04c main
[junit4:junit4] 2> 648099 T1331 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 648099 T1381 C389 P15050 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648100 T1309 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a3cb04c main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):C1)}
[junit4:junit4] 2> 648101 T1381 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 648101 T1318 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 648102 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648104 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648108 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648110 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648112 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648115 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648118 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648119 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648121 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648126 T1315 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670127&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648127 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670127&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648132 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670127&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648133 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648138 T1385 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670139&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648140 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670139&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648143 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670139&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648144 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648145 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648152 T1386 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670153&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648154 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670153&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648158 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670153&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648159 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648164 T1387 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670165&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648166 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670165&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648169 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670165&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648170 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648176 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648179 T1277 C392 P17142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_4,generation=4}
[junit4:junit4] 2> 648179 T1277 C392 P17142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 648181 T1277 C392 P17142 oass.SolrIndexSearcher.<init> Opening Searcher@108019f main
[junit4:junit4] 2> 648182 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648182 T1291 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@108019f main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 648183 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439181748490469376)],commit=} 0 10
[junit4:junit4] 2> 648197 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:15050/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439181748507246592)]} 0 3
[junit4:junit4] 2> 648198 T1389 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:18829/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439181748507246592)]} 0 9
[junit4:junit4] 2> 648198 T1330 C388 P18829 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:15050/collection1/, StdNode: http://127.0.0.1:18829/collection1/, StdNode: http://127.0.0.1:22580/collection1/, StdNode: http://127.0.0.1:37001/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 648201 T1332 C388 P18829 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648201 T1393 C391 P22580 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648201 T1394 C390 P37001 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648202 T1389 C389 P15050 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648202 T1393 C391 P22580 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648202 T1394 C390 P37001 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648203 T1393 C391 P22580 oass.SolrIndexSearcher.<init> Opening Searcher@57509da0 main
[junit4:junit4] 2> 648203 T1394 C390 P37001 oass.SolrIndexSearcher.<init> Opening Searcher@2213b480 main
[junit4:junit4] 2> 648204 T1332 C388 P18829 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_4,generation=4}
[junit4:junit4] 2> 648205 T1325 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57509da0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648204 T1393 C391 P22580 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648205 T1389 C389 P15050 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_4,generation=4}
[junit4:junit4] 2> 648205 T1359 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2213b480 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648205 T1332 C388 P18829 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 648204 T1394 C390 P37001 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648206 T1389 C389 P15050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 648205 T1393 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 648208 T1394 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 648209 T1332 C388 P18829 oass.SolrIndexSearcher.<init> Opening Searcher@3d0c7242 main
[junit4:junit4] 2> 648210 T1389 C389 P15050 oass.SolrIndexSearcher.<init> Opening Searcher@d5a5e36 main
[junit4:junit4] 2> 648210 T1332 C388 P18829 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648211 T1389 C389 P15050 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648211 T1341 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d0c7242 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 648211 T1309 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d5a5e36 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):C1)}
[junit4:junit4] 2> 648212 T1332 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 648212 T1389 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 648213 T1330 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 648214 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648216 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648220 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648222 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648224 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648227 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648230 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648231 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648233 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648239 T1333 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670240&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648240 T1315 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670240&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648244 T1333 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670240&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648245 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648251 T1334 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670252&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648252 T1385 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670252&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648257 T1334 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670252&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 648258 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648259 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648265 T1335 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670266&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648266 T1386 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670266&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648270 T1335 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670266&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648271 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648276 T1387 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670277&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648277 T1395 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670277&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648280 T1395 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670277&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648281 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 648287 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648290 T1277 C392 P17142 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17129257 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56b08935),segFN=segments_5,generation=5}
[junit4:junit4] 2> 648291 T1277 C392 P17142 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 648292 T1277 C392 P17142 oass.SolrIndexSearcher.<init> Opening Searcher@6a40d7a4 main
[junit4:junit4] 2> 648293 T1277 C392 P17142 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648293 T1291 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a40d7a4 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 648294 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439181748606861312)],commit=} 0 10
[junit4:junit4] 2> 648309 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:15050/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439181748624687104)]} 0 3
[junit4:junit4] 2> 648309 T1301 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:37001/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439181748624687104)]} 0 8
[junit4:junit4] 2> 648310 T1348 C390 P37001 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:15050/collection1/, StdNode: http://127.0.0.1:18829/collection1/, StdNode: http://127.0.0.1:22580/collection1/, StdNode: http://127.0.0.1:37001/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 648312 T1398 C388 P18829 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648312 T1399 C390 P37001 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648313 T1399 C390 P37001 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648313 T1317 C391 P22580 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648313 T1399 C390 P37001 oass.SolrIndexSearcher.<init> Opening Searcher@6ec394c7 main
[junit4:junit4] 2> 648313 T1301 C389 P15050 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 648314 T1399 C390 P37001 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648314 T1317 C391 P22580 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 648315 T1398 C388 P18829 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@61099381 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ac221b4),segFN=segments_5,generation=5}
[junit4:junit4] 2> 648315 T1359 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ec394c7 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648316 T1398 C388 P18829 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 648315 T1317 C391 P22580 oass.SolrIndexSearcher.<init> Opening Searcher@6e0a2df6 main
[junit4:junit4] 2> 648316 T1399 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 648317 T1317 C391 P22580 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648317 T1325 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e0a2df6 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 648317 T1398 C388 P18829 oass.SolrIndexSearcher.<init> Opening Searcher@3fb14bce main
[junit4:junit4] 2> 648317 T1301 C389 P15050 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50dcb241 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5c675382),segFN=segments_5,generation=5}
[junit4:junit4] 2> 648318 T1398 C388 P18829 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648318 T1317 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 648319 T1341 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3fb14bce main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 648319 T1301 C389 P15050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 648320 T1398 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 648321 T1301 C389 P15050 oass.SolrIndexSearcher.<init> Opening Searcher@1b79db3c main
[junit4:junit4] 2> 648322 T1301 C389 P15050 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 648323 T1309 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b79db3c main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):C1)}
[junit4:junit4] 2> 648323 T1301 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 648324 T1348 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 26
[junit4:junit4] 2> 648325 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648327 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648331 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648334 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 648335 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648338 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648341 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648342 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648344 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648350 T1350 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670350&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648351 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670350&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648355 T1297 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670350&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648356 T1302 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648363 T1351 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670363&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648363 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670363&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648367 T1299 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670363&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648368 T1330 C388 P18829 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648369 T1265 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 648376 T1352 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670376&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648376 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670376&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648381 T1376 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670376&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648382 T1318 C391 P22580 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 648388 T1380 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670388&shard.url=127.0.0.1:22580/collection1/|127.0.0.1:37001/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 648388 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670388&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 648392 T1379 C389 P15050 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372510670388&shard.url=127.0.0.1:15050/collection1/|127.0.0.1:18829/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 648393 T1348 C390 P37001 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 648401 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439181748724301824)]} 0 5
[junit4:junit4] 2> 648415 T1329 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:15050/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439181748731641856)]} 0 5
[junit4:junit4] 2> 648415 T1400 C389 P15050 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439181748731641856)]} 0 12
[junit4:junit4] 2> 648421 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1439181748748419072)]} 0 2
[junit4:junit4] 2> 648434 T1347 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:22580/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439181748752613376)]} 0 4
[junit4:junit4] 2> 648435 T1401 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1439181748752613376)]} 0 12
[junit4:junit4] 2> 648441 T1277 C392 P17142 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1439181748769390592)]} 0 2
[junit4:junit4] 2> 648452 T1347 C390 P37001 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:22580/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439181748775682048)]} 0 2
[junit4:junit4] 2> 648453 T1393 C391 P22580 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:18829/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1439181748775682048)]} 0 8
[junit4:junit4] 2> 648454 T1402 C388 P18829 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3]} 0 11
[junit4:junit4] 2> 648459
[...truncated too long message...]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Asserting, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_LY, timezone=SystemV/EST5EDT
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=150576424,total=469762048
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ChaosMonkeyNothingIsSafeTest, TestSerializedLuceneMatchVersion, ResponseLogComponentTest, TestStressVersions, TestFastLRUCache, TestFieldResource, TestDefaultSearchFieldResource, SOLR749Test, DirectUpdateHandlerOptimizeTest, ClusterStateUpdateTest, FieldMutatingUpdateProcessorTest, TestSort, SoftAutoCommitTest, RAMDirectoryFactoryTest, SimpleFacetsTest, TermVectorComponentTest, TestMaxScoreQueryParser, QueryParsingTest, TestGroupingSearch, RecoveryZkTest, TestSolrXml, MoreLikeThisHandlerTest, SolrInfoMBeanTest, PeerSyncTest, AutoCommitTest, BasicHdfsTest, TestDocSet, ReturnFieldsTest, TestFaceting, ShardRoutingTest, TestDynamicFieldResource, TestPropInjectDefaults, PreAnalyzedUpdateProcessorTest, TestSolrQueryParser, ResourceLoaderTest, SchemaVersionSpecificBehaviorTest, JsonLoaderTest, SolrIndexSplitterTest, DocValuesTest, TestShardHandlerFactory, LegacyHTMLStripCharFilterTest, TestReversedWildcardFilterFactory, ZkCLITest, TestStressRecovery, TestWriterPerf, TestCharFilters, DocumentBuilderTest, TestBadConfig, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=3F36C5300C3BE0C4 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_LY -Dtests.timezone=SystemV/EST5EDT -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([3F36C5300C3BE0C4]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=1717, name=recoveryCmdExecutor-576-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([3F36C5300C3BE0C4]:0)
[junit4:junit4] Completed on J0 in 241.83s, 1 test, 2 errors <<< FAILURES!
[...truncated 648 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 745 ignored (4 assumptions)
Total time: 38 minutes 27 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4097 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4097/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([93B32AAE401ED3E]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([93B32AAE401ED3E]:0)
Build Log:
[...truncated 9422 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 665421 T1644 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 665428 T1644 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372481514129
[junit4:junit4] 2> 665430 T1644 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 665431 T1645 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 665532 T1644 oasc.ZkTestServer.run start zk server on port:10096
[junit4:junit4] 2> 665533 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 665537 T1651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2644028b name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 665537 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 665538 T1644 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 665550 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 665552 T1653 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@604e5512 name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 665552 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 665553 T1644 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 665572 T1644 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 665575 T1644 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 665577 T1644 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 665586 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 665587 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 665600 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 665601 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 665705 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 665706 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 665709 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 665710 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 665723 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 665724 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 665727 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 665728 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 665731 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 665731 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 665734 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 665735 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 665738 T1644 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 665739 T1644 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 666027 T1644 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 666030 T1644 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10099
[junit4:junit4] 2> 666031 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 666031 T1644 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 666032 T1644 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458
[junit4:junit4] 2> 666032 T1644 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/solr.xml
[junit4:junit4] 2> 666033 T1644 oasc.CoreContainer.<init> New CoreContainer 473372290
[junit4:junit4] 2> 666034 T1644 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/'
[junit4:junit4] 2> 666034 T1644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/'
[junit4:junit4] 2> 666166 T1644 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 666167 T1644 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 666168 T1644 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 666169 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 666169 T1644 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 666170 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 666170 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 666171 T1644 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 666171 T1644 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 666172 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 666193 T1644 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 666194 T1644 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10096/solr
[junit4:junit4] 2> 666194 T1644 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 666195 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 666202 T1664 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1abca670 name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 666203 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 666236 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 666249 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 666252 T1666 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ab64283 name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 666252 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 666259 T1644 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 666273 T1644 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 666276 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 666283 T1644 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10099_
[junit4:junit4] 2> 666300 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10099_
[junit4:junit4] 2> 666304 T1644 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 666325 T1644 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 666328 T1644 oasc.Overseer.start Overseer (id=89946948510482435-127.0.0.1:10099_-n_0000000000) starting
[junit4:junit4] 2> 666331 T1644 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 666335 T1668 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 666335 T1644 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 666338 T1644 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 666340 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 666342 T1667 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 666347 T1669 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 666347 T1669 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 666349 T1669 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 667846 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 667847 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10099_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10099"}
[junit4:junit4] 2> 667848 T1667 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 667848 T1667 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 667886 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 668351 T1669 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 668351 T1669 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/collection1
[junit4:junit4] 2> 668351 T1669 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 668352 T1669 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 668353 T1669 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 668355 T1669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/collection1/'
[junit4:junit4] 2> 668357 T1669 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 668358 T1669 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/collection1/lib/README' to classloader
[junit4:junit4] 2> 668442 T1669 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 668531 T1669 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 668633 T1669 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 668644 T1669 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 669525 T1669 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 669545 T1669 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 669552 T1669 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 669572 T1669 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 669579 T1669 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 669586 T1669 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 669588 T1669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 669588 T1669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 669588 T1669 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 669590 T1669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 669590 T1669 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 669590 T1669 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 669591 T1669 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372481514458/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/control/data/
[junit4:junit4] 2> 669591 T1669 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@265c517a
[junit4:junit4] 2> 669595 T1669 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/control/data
[junit4:junit4] 2> 669595 T1669 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/control/data/index/
[junit4:junit4] 2> 669596 T1669 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 669596 T1669 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/control/data/index
[junit4:junit4] 2> 669603 T1669 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_1,generation=1}
[junit4:junit4] 2> 669603 T1669 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 669606 T1669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 669607 T1669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 669608 T1669 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 669608 T1669 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 669609 T1669 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 669609 T1669 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 669610 T1669 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 669610 T1669 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 669611 T1669 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 669634 T1669 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 669645 T1669 oass.SolrIndexSearcher.<init> Opening Searcher@3d359b8 main
[junit4:junit4] 2> 669647 T1669 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 669647 T1669 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 669651 T1670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d359b8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 669653 T1669 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 669653 T1669 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10099 collection:control_collection shard:shard1
[junit4:junit4] 2> 669654 T1669 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 669680 T1669 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 669695 T1669 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 669695 T1669 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 669696 T1669 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10099/collection1/
[junit4:junit4] 2> 669696 T1669 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 669696 T1669 oasc.SyncStrategy.syncToMe http://127.0.0.1:10099/collection1/ has no replicas
[junit4:junit4] 2> 669696 T1669 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10099/collection1/ shard1
[junit4:junit4] 2> 669697 T1669 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 670891 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 670910 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 670941 T1669 oasc.ZkController.register We are http://127.0.0.1:10099/collection1/ and leader is http://127.0.0.1:10099/collection1/
[junit4:junit4] 2> 670941 T1669 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10099
[junit4:junit4] 2> 670941 T1669 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 670941 T1669 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 670942 T1669 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 670947 T1669 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 670949 T1644 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 670949 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 670950 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 670967 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 670969 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 670977 T1673 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ee9cf09 name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 670977 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 670978 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 670981 T1644 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 671259 T1644 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 671263 T1644 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10107
[junit4:junit4] 2> 671264 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 671264 T1644 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 671265 T1644 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681
[junit4:junit4] 2> 671265 T1644 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/solr.xml
[junit4:junit4] 2> 671266 T1644 oasc.CoreContainer.<init> New CoreContainer 809135128
[junit4:junit4] 2> 671266 T1644 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/'
[junit4:junit4] 2> 671267 T1644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/'
[junit4:junit4] 2> 671401 T1644 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 671402 T1644 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 671403 T1644 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 671403 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 671404 T1644 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 671404 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 671405 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 671405 T1644 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 671406 T1644 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 671406 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 671423 T1644 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 671424 T1644 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10096/solr
[junit4:junit4] 2> 671424 T1644 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 671425 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 671428 T1684 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20f4b44c name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 671429 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 671432 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 671444 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 671446 T1686 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@496fd6ca name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 671446 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 671451 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 672418 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 672419 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10099_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10099"}
[junit4:junit4] 2> 672446 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 672446 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 672446 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 672457 T1644 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10107_
[junit4:junit4] 2> 672459 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10107_
[junit4:junit4] 2> 672462 T1673 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 672462 T1686 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 672462 T1666 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 672463 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 672462 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 672463 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 672475 T1687 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 672475 T1687 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 672477 T1687 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 673960 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 673961 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10107_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10107"}
[junit4:junit4] 2> 673962 T1667 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 673962 T1667 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 673983 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 673983 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 673983 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 674479 T1687 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 674479 T1687 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/collection1
[junit4:junit4] 2> 674479 T1687 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 674480 T1687 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 674481 T1687 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 674482 T1687 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/collection1/'
[junit4:junit4] 2> 674484 T1687 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/collection1/lib/README' to classloader
[junit4:junit4] 2> 674484 T1687 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 674539 T1687 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 674601 T1687 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 674703 T1687 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 674710 T1687 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 675359 T1687 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 675375 T1687 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 675380 T1687 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 675399 T1687 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 675405 T1687 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 675410 T1687 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 675411 T1687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 675412 T1687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 675412 T1687 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 675413 T1687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 675413 T1687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 675414 T1687 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 675414 T1687 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372481519681/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty1/
[junit4:junit4] 2> 675414 T1687 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@265c517a
[junit4:junit4] 2> 675417 T1687 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty1
[junit4:junit4] 2> 675418 T1687 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty1/index/
[junit4:junit4] 2> 675418 T1687 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 675419 T1687 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty1/index
[junit4:junit4] 2> 675423 T1687 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 675423 T1687 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 675427 T1687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 675428 T1687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 675428 T1687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 675429 T1687 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 675430 T1687 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 675430 T1687 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 675431 T1687 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 675431 T1687 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 675432 T1687 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 675449 T1687 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 675458 T1687 oass.SolrIndexSearcher.<init> Opening Searcher@4a577f main
[junit4:junit4] 2> 675460 T1687 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 675460 T1687 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 675465 T1688 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4a577f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 675468 T1687 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 675468 T1687 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10107 collection:collection1 shard:shard1
[junit4:junit4] 2> 675469 T1687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 675496 T1687 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 675500 T1687 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 675500 T1687 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 675501 T1687 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10107/collection1/
[junit4:junit4] 2> 675501 T1687 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 675501 T1687 oasc.SyncStrategy.syncToMe http://127.0.0.1:10107/collection1/ has no replicas
[junit4:junit4] 2> 675501 T1687 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10107/collection1/ shard1
[junit4:junit4] 2> 675502 T1687 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 676998 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 677019 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 677019 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 677019 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 677060 T1687 oasc.ZkController.register We are http://127.0.0.1:10107/collection1/ and leader is http://127.0.0.1:10107/collection1/
[junit4:junit4] 2> 677060 T1687 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10107
[junit4:junit4] 2> 677060 T1687 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 677061 T1687 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 677061 T1687 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 677063 T1687 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 677064 T1644 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 677064 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 677065 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 677374 T1644 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 677393 T1644 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10113
[junit4:junit4] 2> 677395 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 677396 T1644 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 677396 T1644 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782
[junit4:junit4] 2> 677397 T1644 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/solr.xml
[junit4:junit4] 2> 677398 T1644 oasc.CoreContainer.<init> New CoreContainer 1613064237
[junit4:junit4] 2> 677399 T1644 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/'
[junit4:junit4] 2> 677400 T1644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/'
[junit4:junit4] 2> 677543 T1644 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 677544 T1644 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 677544 T1644 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 677545 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 677545 T1644 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 677546 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 677546 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 677547 T1644 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 677547 T1644 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 677548 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 677564 T1644 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 677565 T1644 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10096/solr
[junit4:junit4] 2> 677565 T1644 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 677566 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 677569 T1700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38f8ae2b name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 677570 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 677572 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 677584 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 677613 T1702 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@439c32dd name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 677613 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 677653 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 678524 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 678525 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10107_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10107"}
[junit4:junit4] 2> 678543 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 678543 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 678544 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 678543 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 678657 T1644 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10113_
[junit4:junit4] 2> 678659 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10113_
[junit4:junit4] 2> 678662 T1666 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 678662 T1673 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 678662 T1702 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 678662 T1686 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 678662 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 678662 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 678662 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 678663 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 678675 T1703 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 678676 T1703 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 678677 T1703 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 680047 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680048 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10113_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10113"}
[junit4:junit4] 2> 680049 T1667 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 680049 T1667 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 680066 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 680066 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 680066 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 680066 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 680679 T1703 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 680679 T1703 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/collection1
[junit4:junit4] 2> 680679 T1703 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 680680 T1703 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 680681 T1703 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 680682 T1703 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/collection1/'
[junit4:junit4] 2> 680684 T1703 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/collection1/lib/README' to classloader
[junit4:junit4] 2> 680684 T1703 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 680741 T1703 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 680806 T1703 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 680907 T1703 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 680915 T1703 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 681550 T1703 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 681564 T1703 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 681568 T1703 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 681584 T1703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 681589 T1703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 681594 T1703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 681595 T1703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 681596 T1703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 681596 T1703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 681597 T1703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 681598 T1703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 681598 T1703 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 681598 T1703 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372481525782/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty2/
[junit4:junit4] 2> 681599 T1703 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@265c517a
[junit4:junit4] 2> 681602 T1703 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty2
[junit4:junit4] 2> 681602 T1703 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty2/index/
[junit4:junit4] 2> 681603 T1703 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 681603 T1703 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty2/index
[junit4:junit4] 2> 681607 T1703 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2273e0a1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c44ed23),segFN=segments_1,generation=1}
[junit4:junit4] 2> 681607 T1703 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 681611 T1703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 681611 T1703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 681612 T1703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 681613 T1703 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 681613 T1703 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 681613 T1703 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 681614 T1703 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 681615 T1703 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 681615 T1703 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 681632 T1703 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 681640 T1703 oass.SolrIndexSearcher.<init> Opening Searcher@7caf2827 main
[junit4:junit4] 2> 681641 T1703 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 681641 T1703 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 681646 T1704 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7caf2827 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 681649 T1703 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 681649 T1703 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10113 collection:collection1 shard:shard2
[junit4:junit4] 2> 681650 T1703 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 681671 T1703 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 681673 T1703 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 681674 T1703 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 681674 T1703 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10113/collection1/
[junit4:junit4] 2> 681674 T1703 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 681675 T1703 oasc.SyncStrategy.syncToMe http://127.0.0.1:10113/collection1/ has no replicas
[junit4:junit4] 2> 681675 T1703 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10113/collection1/ shard2
[junit4:junit4] 2> 681675 T1703 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 683073 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 683112 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 683112 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 683112 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 683112 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 683159 T1703 oasc.ZkController.register We are http://127.0.0.1:10113/collection1/ and leader is http://127.0.0.1:10113/collection1/
[junit4:junit4] 2> 683159 T1703 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10113
[junit4:junit4] 2> 683159 T1703 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 683159 T1703 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 683160 T1703 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 683162 T1703 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 683163 T1644 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 683164 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 683165 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 683461 T1644 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 683464 T1644 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10122
[junit4:junit4] 2> 683465 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 683466 T1644 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 683466 T1644 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883
[junit4:junit4] 2> 683467 T1644 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/solr.xml
[junit4:junit4] 2> 683467 T1644 oasc.CoreContainer.<init> New CoreContainer 1477930414
[junit4:junit4] 2> 683468 T1644 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/'
[junit4:junit4] 2> 683469 T1644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/'
[junit4:junit4] 2> 683604 T1644 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 683605 T1644 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 683605 T1644 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 683606 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 683606 T1644 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 683607 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 683607 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 683608 T1644 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 683608 T1644 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 683609 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 683627 T1644 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 683628 T1644 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10096/solr
[junit4:junit4] 2> 683628 T1644 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 683629 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 683634 T1716 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d494a6f name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 683634 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 683637 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 683650 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 683652 T1718 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36297348 name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 683652 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 683657 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 684628 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 684629 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10113_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10113"}
[junit4:junit4] 2> 684643 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684643 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684643 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684643 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684643 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684660 T1644 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10122_
[junit4:junit4] 2> 684662 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10122_
[junit4:junit4] 2> 684663 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 684664 T1673 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 684664 T1702 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 684664 T1686 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 684664 T1666 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 684664 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 684664 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 684664 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 684665 T1718 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 684665 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 684670 T1719 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 684671 T1719 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 684672 T1719 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 686156 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 686157 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10122_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10122"}
[junit4:junit4] 2> 686158 T1667 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 686158 T1667 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 686163 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 686163 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 686163 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 686163 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 686163 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 686674 T1719 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 686674 T1719 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/collection1
[junit4:junit4] 2> 686674 T1719 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 686675 T1719 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 686676 T1719 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 686677 T1719 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/collection1/'
[junit4:junit4] 2> 686679 T1719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 686679 T1719 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/collection1/lib/README' to classloader
[junit4:junit4] 2> 686734 T1719 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 686796 T1719 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 686898 T1719 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 686905 T1719 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 687581 T1719 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 687594 T1719 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 687599 T1719 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 687614 T1719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 687620 T1719 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 687625 T1719 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 687626 T1719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 687627 T1719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 687627 T1719 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 687628 T1719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 687629 T1719 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 687629 T1719 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 687630 T1719 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372481531883/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty3/
[junit4:junit4] 2> 687630 T1719 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@265c517a
[junit4:junit4] 2> 687633 T1719 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty3
[junit4:junit4] 2> 687633 T1719 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty3/index/
[junit4:junit4] 2> 687634 T1719 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 687634 T1719 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty3/index
[junit4:junit4] 2> 687638 T1719 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 687638 T1719 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 687642 T1719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 687642 T1719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 687643 T1719 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 687643 T1719 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 687644 T1719 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 687644 T1719 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 687645 T1719 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 687646 T1719 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 687646 T1719 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 687663 T1719 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 687671 T1719 oass.SolrIndexSearcher.<init> Opening Searcher@6a58aed0 main
[junit4:junit4] 2> 687673 T1719 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 687673 T1719 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 687678 T1720 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a58aed0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 687680 T1719 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 687681 T1719 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10122 collection:collection1 shard:shard1
[junit4:junit4] 2> 687691 T1719 oasc.ZkController.register We are http://127.0.0.1:10122/collection1/ and leader is http://127.0.0.1:10107/collection1/
[junit4:junit4] 2> 687691 T1719 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10122
[junit4:junit4] 2> 687691 T1719 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 687692 T1719 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C217 name=collection1 org.apache.solr.core.SolrCore@5aabb2b0 url=http://127.0.0.1:10122/collection1 node=127.0.0.1:10122_ C217_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:10122_, base_url=http://127.0.0.1:10122}
[junit4:junit4] 2> 687692 T1721 C217 P10122 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 687692 T1719 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 687693 T1721 C217 P10122 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 687693 T1721 C217 P10122 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 687694 T1721 C217 P10122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 687694 T1644 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 687695 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 687695 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 687713 T1721 C217 P10122 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 687728 T1676 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 687987 T1644 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 687990 T1644 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10126
[junit4:junit4] 2> 687991 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 687992 T1644 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 687992 T1644 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414
[junit4:junit4] 2> 687993 T1644 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/solr.xml
[junit4:junit4] 2> 687993 T1644 oasc.CoreContainer.<init> New CoreContainer 351057261
[junit4:junit4] 2> 687994 T1644 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/'
[junit4:junit4] 2> 687995 T1644 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/'
[junit4:junit4] 2> 688128 T1644 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 688129 T1644 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 688130 T1644 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 688130 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 688131 T1644 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 688131 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 688132 T1644 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 688132 T1644 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 688133 T1644 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 688133 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 688151 T1644 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 688152 T1644 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10096/solr
[junit4:junit4] 2> 688152 T1644 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 688153 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 688156 T1733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fd1cde8 name:ZooKeeperConnection Watcher:127.0.0.1:10096 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 688157 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 688159 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 688172 T1644 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 688174 T1735 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@231ac3a4 name:ZooKeeperConnection Watcher:127.0.0.1:10096/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 688174 T1644 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 688197 T1644 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 689192 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 689193 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10122_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10122"}
[junit4:junit4] 2> 689210 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689210 T1644 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10126_
[junit4:junit4] 2> 689210 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689210 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689210 T1735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689210 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689210 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689212 T1644 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10126_
[junit4:junit4] 2> 689215 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689215 T1735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 689215 T1673 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689215 T1686 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689216 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 689215 T1666 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689215 T1702 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689216 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 689217 T1735 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689216 T1718 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 689216 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 689217 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 689224 T1736 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 689224 T1736 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 689225 T1736 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 689730 T1676 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 689730 T1676 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:10122_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 690714 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 690715 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10126_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10126"}
[junit4:junit4] 2> 690716 T1667 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 690716 T1667 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 690728 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 690728 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 690728 T1735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 690728 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 690728 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 690728 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 691228 T1736 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 691228 T1736 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/collection1
[junit4:junit4] 2> 691228 T1736 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 691229 T1736 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 691230 T1736 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 691231 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/collection1/'
[junit4:junit4] 2> 691233 T1736 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/collection1/lib/README' to classloader
[junit4:junit4] 2> 691233 T1736 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 691287 T1736 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 691349 T1736 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 691451 T1736 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 691458 T1736 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C218 name=collection1 org.apache.solr.core.SolrCore@5aabb2b0 url=http://127.0.0.1:10122/collection1 node=127.0.0.1:10122_ C218_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:10122_, base_url=http://127.0.0.1:10122}
[junit4:junit4] 2> 691732 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10107/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 691732 T1721 C218 P10122 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10122 START replicas=[http://127.0.0.1:10107/collection1/] nUpdates=100
[junit4:junit4] 2> 691733 T1721 C218 P10122 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 691734 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 691734 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 691734 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 691734 T1721 C218 P10122 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 691735 T1721 C218 P10122 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10107/collection1/. core=collection1
[junit4:junit4] 2> 691735 T1721 C218 P10122 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C219 name=collection1 org.apache.solr.core.SolrCore@5e2ce59 url=http://127.0.0.1:10107/collection1 node=127.0.0.1:10107_ C219_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10107_, base_url=http://127.0.0.1:10107, leader=true}
[junit4:junit4] 2> 691743 T1677 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 691749 T1678 C219 P10107 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 691752 T1678 C219 P10107 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 691753 T1678 C219 P10107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 691754 T1678 C219 P10107 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 691754 T1678 C219 P10107 oass.SolrIndexSearcher.<init> Opening Searcher@274eb181 realtime
[junit4:junit4] 2> 691755 T1678 C219 P10107 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 691755 T1678 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 691757 T1721 C218 P10122 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 691757 T1721 C218 P10122 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 691760 T1679 C219 P10107 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 691760 T1679 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 691761 T1721 C218 P10122 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 691761 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 691761 T1721 C218 P10122 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 691762 T1721 C218 P10122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 691763 T1721 C218 P10122 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 692097 T1736 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 692109 T1736 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 692113 T1736 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 692129 T1736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 692134 T1736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 692138 T1736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 692139 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 692140 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 692140 T1736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 692142 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 692142 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 692142 T1736 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 692142 T1736 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372481536414/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty4/
[junit4:junit4] 2> 692143 T1736 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@265c517a
[junit4:junit4] 2> 692145 T1736 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty4
[junit4:junit4] 2> 692146 T1736 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty4/index/
[junit4:junit4] 2> 692146 T1736 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 692146 T1736 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372481514129/jetty4/index
[junit4:junit4] 2> 692150 T1736 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d19f259 lockFactory=org.apache.lucene.store.NativeFSLockFactory@59e13da0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 692150 T1736 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 692153 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 692154 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 692154 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 692155 T1736 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 692156 T1736 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 692156 T1736 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 692157 T1736 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 692158 T1736 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 692158 T1736 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 692174 T1736 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 692181 T1736 oass.SolrIndexSearcher.<init> Opening Searcher@52cbd8c1 main
[junit4:junit4] 2> 692183 T1736 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 692183 T1736 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 692187 T1738 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52cbd8c1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 692190 T1736 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 692190 T1736 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10126 collection:collection1 shard:shard2
[junit4:junit4] 2> 692193 T1736 oasc.ZkController.register We are http://127.0.0.1:10126/collection1/ and leader is http://127.0.0.1:10113/collection1/
[junit4:junit4] 2> 692193 T1736 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10126
[junit4:junit4] 2> 692194 T1736 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 692194 T1736 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C220 name=collection1 org.apache.solr.core.SolrCore@11470cee url=http://127.0.0.1:10126/collection1 node=127.0.0.1:10126_ C220_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:10126_, base_url=http://127.0.0.1:10126}
[junit4:junit4] 2> 692194 T1739 C220 P10126 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 692195 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 692195 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 692195 T1739 C220 P10126 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 692196 T1739 C220 P10126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 692196 T1644 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 692197 T1644 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 692197 T1739 C220 P10126 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 692197 T1644 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 692216 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 692217 T1692 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 692218 T1644 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 692219 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 692238 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 692239 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10122_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10122"}
[junit4:junit4] 2> 692244 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10126_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10126"}
[junit4:junit4] 2> 692256 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 692256 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 692256 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 692256 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 692256 T1735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 692256 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 693219 T1692 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 693219 T1692 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:10126_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 693221 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 694223 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C220_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:10126_, base_url=http://127.0.0.1:10126}
[junit4:junit4] 2> 695221 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:10113/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 695221 T1739 C220 P10126 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:10126 START replicas=[http://127.0.0.1:10113/collection1/] nUpdates=100
[junit4:junit4] 2> 695222 T1739 C220 P10126 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 695222 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 695222 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 695222 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 695223 T1739 C220 P10126 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 695223 T1739 C220 P10126 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:10113/collection1/. core=collection1
[junit4:junit4] 2> 695223 T1739 C220 P10126 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 695226 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C221 name=collection1 org.apache.solr.core.SolrCore@7fb5ae71 url=http://127.0.0.1:10113/collection1 node=127.0.0.1:10113_ C221_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10113_, base_url=http://127.0.0.1:10113, leader=true}
[junit4:junit4] 2> 695232 T1693 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 695238 T1694 C221 P10113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 695242 T1694 C221 P10113 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2273e0a1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c44ed23),segFN=segments_1,generation=1}
[junit4:junit4] 2> 695242 T1694 C221 P10113 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 695243 T1694 C221 P10113 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 695244 T1694 C221 P10113 oass.SolrIndexSearcher.<init> Opening Searcher@6a02d2c4 realtime
[junit4:junit4] 2> 695244 T1694 C221 P10113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 695245 T1694 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 695246 T1739 C220 P10126 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 695246 T1739 C220 P10126 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 695248 T1695 C221 P10113 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 695248 T1695 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 695249 T1739 C220 P10126 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 695249 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 695249 T1739 C220 P10126 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 695249 T1739 C220 P10126 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 695251 T1739 C220 P10126 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 695272 T1667 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 695273 T1667 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:10126_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:10126"}
[junit4:junit4] 2> 695285 T1718 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 695285 T1666 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 695285 T1673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 695285 T1702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 695285 T1735 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 695285 T1686 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 696228 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696229 T1644 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C222 name=collection1 org.apache.solr.core.SolrCore@3a79526c url=http://127.0.0.1:10099/collection1 node=127.0.0.1:10099_ C222_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10099_, base_url=http://127.0.0.1:10099, leader=true}
[junit4:junit4] 2> 696243 T1656 C222 P10099 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_1,generation=1}
[junit4:junit4] 2> 696244 T1656 C222 P10099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 696246 T1656 C222 P10099 oass.SolrIndexSearcher.<init> Opening Searcher@4d2c00fc realtime
[junit4:junit4] 2> 696246 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439151208465956864)} 0 7
[junit4:junit4] 2> C218_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10122_, base_url=http://127.0.0.1:10122}
[junit4:junit4] 2> 696277 T1708 C218 P10122 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_1,generation=1}
[junit4:junit4] 2> 696278 T1708 C218 P10122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> C220_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:10126_, base_url=http://127.0.0.1:10126}
[junit4:junit4] 2> 696279 T1725 C220 P10126 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6d19f259 lockFactory=org.apache.lucene.store.NativeFSLockFactory@59e13da0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 696280 T1725 C220 P10126 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 696280 T1708 C218 P10122 oass.SolrIndexSearcher.<init> Opening Searcher@3d767138 realtime
[junit4:junit4] 2> 696281 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439151208487976960&update.from=http://127.0.0.1:10107/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439151208487976960)} 0 8
[junit4:junit4] 2> 696281 T1725 C220 P10126 oass.SolrIndexSearcher.<init> Opening Searcher@5f0cf231 realtime
[junit4:junit4] 2> 696282 T1725 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439151208494268416&update.from=http://127.0.0.1:10113/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439151208494268416)} 0 5
[junit4:junit4] 2> 696283 T1696 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439151208494268416)} 0 18
[junit4:junit4] 2> 696284 T1680 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439151208487976960)} 0 24
[junit4:junit4] 2> 696285 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696287 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696290 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696292 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696294 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696296 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696299 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696303 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696307 T1656 C222 P10099 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_2,generation=2}
[junit4:junit4] 2> 696308 T1656 C222 P10099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 696310 T1656 C222 P10099 oass.SolrIndexSearcher.<init> Opening Searcher@3ee3f9bc main
[junit4:junit4] 2> 696311 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696312 T1670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ee3f9bc main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 696312 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439151208532017152)],commit=} 0 10
[junit4:junit4] 2> 696320 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439151208546697216)]} 0 1
[junit4:junit4] 2> 696321 T1681 C219 P10107 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10107/collection1/, StdNode: http://127.0.0.1:10122/collection1/, StdNode: http://127.0.0.1:10113/collection1/, StdNode: http://127.0.0.1:10126/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 696323 T1682 C219 P10107 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696324 T1727 C220 P10126 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696325 T1708 C218 P10122 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696325 T1727 C220 P10126 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696325 T1696 C221 P10113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696326 T1727 C220 P10126 oass.SolrIndexSearcher.<init> Opening Searcher@5ecb5721 main
[junit4:junit4] 2> 696326 T1696 C221 P10113 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696329 T1727 C220 P10126 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696329 T1696 C221 P10113 oass.SolrIndexSearcher.<init> Opening Searcher@7b06a2f9 main
[junit4:junit4] 2> 696330 T1708 C218 P10122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_2,generation=2}
[junit4:junit4] 2> 696330 T1738 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ecb5721 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696329 T1682 C219 P10107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 696331 T1727 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 696331 T1704 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b06a2f9 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696330 T1708 C218 P10122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 696330 T1696 C221 P10113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696331 T1682 C219 P10107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 696333 T1696 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 696334 T1708 C218 P10122 oass.SolrIndexSearcher.<init> Opening Searcher@4cbd12ea main
[junit4:junit4] 2> 696335 T1708 C218 P10122 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696335 T1682 C219 P10107 oass.SolrIndexSearcher.<init> Opening Searcher@5b081828 main
[junit4:junit4] 2> 696336 T1720 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4cbd12ea main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 696337 T1682 C219 P10107 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696337 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 696337 T1688 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b081828 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 696338 T1682 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 696339 T1681 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439151208546697216)],commit=} 0 24
[junit4:junit4] 2> 696340 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696342 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696346 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696351 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696353 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696356 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696359 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696360 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696362 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696369 T1698 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545065&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696369 T1710 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545065&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696373 T1710 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545065&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696374 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 696380 T1692 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545077&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696381 T1711 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545077&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696384 T1711 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545077&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696385 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696387 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696394 T1694 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545090&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696394 T1712 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545090&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696397 T1712 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545090&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696398 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696404 T1755 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545101&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696404 T1713 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545101&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696408 T1713 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545101&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696409 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696414 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696418 T1656 C222 P10099 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_3,generation=3}
[junit4:junit4] 2> 696418 T1656 C222 P10099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 696421 T1656 C222 P10099 oass.SolrIndexSearcher.<init> Opening Searcher@4d861478 main
[junit4:junit4] 2> 696421 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696422 T1670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d861478 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 696423 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439151208647360512)],commit=} 0 11
[junit4:junit4] 2> 696434 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439151208664137728)]} 0 2
[junit4:junit4] 2> 696435 T1676 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10113/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439151208664137728)]} 0 7
[junit4:junit4] 2> 696436 T1697 C221 P10113 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10107/collection1/, StdNode: http://127.0.0.1:10122/collection1/, StdNode: http://127.0.0.1:10113/collection1/, StdNode: http://127.0.0.1:10126/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 696438 T1714 C218 P10122 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696438 T1759 C221 P10113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696438 T1676 C219 P10107 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696441 T1725 C220 P10126 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696441 T1759 C221 P10113 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696442 T1725 C220 P10126 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696443 T1759 C221 P10113 oass.SolrIndexSearcher.<init> Opening Searcher@490aa5bb main
[junit4:junit4] 2> 696443 T1725 C220 P10126 oass.SolrIndexSearcher.<init> Opening Searcher@283b17c6 main
[junit4:junit4] 2> 696444 T1759 C221 P10113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696445 T1725 C220 P10126 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696445 T1714 C218 P10122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_3,generation=3}
[junit4:junit4] 2> 696445 T1704 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@490aa5bb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696447 T1714 C218 P10122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 696446 T1676 C219 P10107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_3,generation=3}
[junit4:junit4] 2> 696446 T1738 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@283b17c6 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696448 T1676 C219 P10107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 696447 T1759 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 696449 T1725 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 696449 T1714 C218 P10122 oass.SolrIndexSearcher.<init> Opening Searcher@51759eb5 main
[junit4:junit4] 2> 696451 T1676 C219 P10107 oass.SolrIndexSearcher.<init> Opening Searcher@164b529 main
[junit4:junit4] 2> 696451 T1714 C218 P10122 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696452 T1676 C219 P10107 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696452 T1720 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51759eb5 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 696453 T1688 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@164b529 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 696454 T1714 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> 696454 T1676 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> 696455 T1697 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 696456 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696458 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696462 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696465 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696467 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696470 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696473 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696474 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696476 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696481 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545179&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696482 T1698 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545179&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696485 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545179&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696486 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 696491 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545189&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696493 T1692 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545189&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696496 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545189&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696497 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696499 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696505 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545202&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696505 T1694 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545202&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696508 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545202&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696509 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 696514 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545212&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696516 T1755 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545212&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696520 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545212&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696521 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 696526 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696530 T1656 C222 P10099 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_4,generation=4}
[junit4:junit4] 2> 696530 T1656 C222 P10099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 696533 T1656 C222 P10099 oass.SolrIndexSearcher.<init> Opening Searcher@7eb0912c main
[junit4:junit4] 2> 696533 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696534 T1670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7eb0912c main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 696536 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439151208764801024)],commit=} 0 12
[junit4:junit4] 2> 696547 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439151208782626816)]} 0 2
[junit4:junit4] 2> 696548 T1764 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10122/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439151208782626816)]} 0 7
[junit4:junit4] 2> 696549 T1709 C218 P10122 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10107/collection1/, StdNode: http://127.0.0.1:10122/collection1/, StdNode: http://127.0.0.1:10113/collection1/, StdNode: http://127.0.0.1:10126/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 696551 T1768 C218 P10122 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696552 T1728 C220 P10126 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696551 T1769 C221 P10113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696551 T1764 C219 P10107 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696554 T1769 C221 P10113 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696553 T1728 C220 P10126 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696556 T1769 C221 P10113 oass.SolrIndexSearcher.<init> Opening Searcher@63b8756c main
[junit4:junit4] 2> 696556 T1728 C220 P10126 oass.SolrIndexSearcher.<init> Opening Searcher@31cd88fe main
[junit4:junit4] 2> 696556 T1769 C221 P10113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696557 T1768 C218 P10122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_4,generation=4}
[junit4:junit4] 2> 696558 T1738 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31cd88fe main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696557 T1728 C220 P10126 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696558 T1764 C219 P10107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_4,generation=4}
[junit4:junit4] 2> 696558 T1768 C218 P10122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 696557 T1704 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@63b8756c main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696559 T1764 C219 P10107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 696558 T1728 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 696560 T1769 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 696561 T1768 C218 P10122 oass.SolrIndexSearcher.<init> Opening Searcher@7d4548e7 main
[junit4:junit4] 2> 696562 T1764 C219 P10107 oass.SolrIndexSearcher.<init> Opening Searcher@7d4d54d3 main
[junit4:junit4] 2> 696563 T1768 C218 P10122 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696563 T1764 C219 P10107 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696563 T1720 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d4548e7 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 696564 T1688 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d4d54d3 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 696565 T1768 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 696565 T1764 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 696566 T1709 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 28
[junit4:junit4] 2> 696567 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696569 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696573 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696576 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696578 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696581 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696584 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696585 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696587 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696592 T1729 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545290&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696594 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545290&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696597 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545290&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696598 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696603 T1730 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545301&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696604 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545301&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696608 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545301&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696609 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696610 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696616 T1731 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545314&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696617 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545314&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696621 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545314&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696622 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696627 T1770 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545325&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696629 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545325&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696632 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545325&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696633 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696638 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696642 T1656 C222 P10099 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2b9a53c7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a4e0809),segFN=segments_5,generation=5}
[junit4:junit4] 2> 696643 T1656 C222 P10099 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 696645 T1656 C222 P10099 oass.SolrIndexSearcher.<init> Opening Searcher@aff7df2 main
[junit4:junit4] 2> 696646 T1656 C222 P10099 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696647 T1670 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@aff7df2 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 696648 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439151208882241536)],commit=} 0 12
[junit4:junit4] 2> 696660 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439151208901115904)]} 0 2
[junit4:junit4] 2> 696661 T1772 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10126/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439151208901115904)]} 0 7
[junit4:junit4] 2> 696662 T1726 C220 P10126 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:10107/collection1/, StdNode: http://127.0.0.1:10122/collection1/, StdNode: http://127.0.0.1:10113/collection1/, StdNode: http://127.0.0.1:10126/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 696665 T1776 C218 P10122 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696665 T1778 C220 P10126 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696665 T1772 C219 P10107 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696665 T1777 C221 P10113 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 696666 T1778 C220 P10126 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696667 T1777 C221 P10113 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 696668 T1778 C220 P10126 oass.SolrIndexSearcher.<init> Opening Searcher@272116ef main
[junit4:junit4] 2> 696668 T1777 C221 P10113 oass.SolrIndexSearcher.<init> Opening Searcher@8defbf0 main
[junit4:junit4] 2> 696669 T1778 C220 P10126 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696669 T1776 C218 P10122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@37eb991e lockFactory=org.apache.lucene.store.NativeFSLockFactory@1047e39d),segFN=segments_5,generation=5}
[junit4:junit4] 2> 696670 T1704 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8defbf0 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696669 T1777 C221 P10113 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696670 T1772 C219 P10107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@45f64269 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5048ed5c),segFN=segments_5,generation=5}
[junit4:junit4] 2> 696670 T1776 C218 P10122 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 696670 T1738 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@272116ef main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 696672 T1772 C219 P10107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 696671 T1777 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 696673 T1778 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 696674 T1776 C218 P10122 oass.SolrIndexSearcher.<init> Opening Searcher@7aad3339 main
[junit4:junit4] 2> 696675 T1772 C219 P10107 oass.SolrIndexSearcher.<init> Opening Searcher@16f3fe08 main
[junit4:junit4] 2> 696675 T1776 C218 P10122 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696676 T1772 C219 P10107 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 696676 T1720 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7aad3339 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 696677 T1688 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16f3fe08 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 696678 T1776 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 696678 T1772 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 696679 T1726 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 28
[junit4:junit4] 2> 696680 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696682 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696686 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696690 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696691 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696694 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696697 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696698 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696700 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696707 T1698 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545403&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696707 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545403&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696711 T1678 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545403&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696712 T1681 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 696719 T1692 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545415&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 696719 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545415&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 696723 T1760 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545415&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696724 T1709 C218 P10122 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 696725 T1644 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 696732 T1694 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545429&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696732 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545429&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696736 T1761 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545429&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 696737 T1697 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 696743 T1755 C221 P10113 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545440&shard.url=127.0.0.1:10113/collection1/|127.0.0.1:10126/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 696743 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545440&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 696748 T1762 C219 P10107 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372481545440&shard.url=127.0.0.1:10107/collection1/|127.0.0.1:10122/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 696749 T1726 C220 P10126 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 696756 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439151209003876352)]} 0 4
[junit4:junit4] 2> 696771 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439151209013313536)]} 0 4
[junit4:junit4] 2> 696772 T1676 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10113/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439151209013313536)]} 0 11
[junit4:junit4] 2> 696773 T1779 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 15
[junit4:junit4] 2> 696778 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1439151209029042176)]} 0 2
[junit4:junit4] 2> 696792 T1725 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10113/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439151209037430784)]} 0 3
[junit4:junit4] 2> 696792 T1696 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1439151209037430784)]} 0 8
[junit4:junit4] 2> 696793 T1680 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2]} 0 12
[junit4:junit4] 2> 696797 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1439151209050013696)]} 0 1
[junit4:junit4] 2> 696809 T1725 C220 P10126 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10113/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439151209058402304)]} 0 1
[junit4:junit4] 2> 696810 T1696 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1439151209058402304)]} 0 6
[junit4:junit4] 2> 696811 T1680 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3]} 0 11
[junit4:junit4] 2> 696815 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1439151209068888064)]} 0 1
[junit4:junit4] 2> 696826 T1708 C218 P10122 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10107/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1439151209076228096)]} 0 1
[junit4:junit4] 2> 696827 T1676 C219 P10107 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:10113/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1439151209076228096)]} 0 6
[junit4:junit4] 2> 696828 T1779 C221 P10113 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4]} 0 10
[junit4:junit4] 2> 696832 T1656 C222 P10099 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1439151209086713856)]} 0 1
[junit4:jun
[...truncated too long message...]
4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=en_US, timezone=HST
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=151178704,total=436207616
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SolrInfoMBeanTest, DirectSolrConnectionTest, TestCoreContainer, AnalysisAfterCoreReloadTest, JsonLoaderTest, RecoveryZkTest, TestFieldResource, DistributedQueryElevationComponentTest, TestSolrCoreProperties, DocValuesTest, BasicHdfsTest, ClusterStateUpdateTest, RAMDirectoryFactoryTest, PluginInfoTest, OverseerTest, CoreAdminHandlerTest, MinimalSchemaTest, TestSurroundQueryParser, ZkControllerTest, DateFieldTest, ShowFileRequestHandlerTest, TestPropInjectDefaults, TestSerializedLuceneMatchVersion, TestFieldTypeCollectionResource, ResponseLogComponentTest, CacheHeaderTest, TestFieldTypeResource, TestPseudoReturnFields, TestFaceting, TestJoin, TestSolrIndexConfig, PrimUtilsTest, TestSearchPerf, PreAnalyzedUpdateProcessorTest, AliasIntegrationTest, QueryElevationComponentTest, TermVectorComponentTest, LoggingHandlerTest, TestTrie, TestDynamicFieldCollectionResource, BinaryUpdateRequestHandlerTest, ChaosMonkeyNothingIsSafeTest, LeaderElectionTest, TestCharFilters, SpatialFilterTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=93B32AAE401ED3E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_US -Dtests.timezone=HST -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([93B32AAE401ED3E]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2093, name=recoveryCmdExecutor-717-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([93B32AAE401ED3E]:0)
[junit4:junit4] Completed on J0 in 213.34s, 1 test, 2 errors <<< FAILURES!
[...truncated 669 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 812 ignored
Total time: 45 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4096 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4096/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([7423CCDB0A7E2DF7]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([7423CCDB0A7E2DF7]:0)
Build Log:
[...truncated 9594 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 653300 T2064 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 653307 T2064 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZkTest-1372453975060
[junit4:junit4] 2> 653309 T2064 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 653310 T2065 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 653410 T2064 oasc.ZkTestServer.run start zk server on port:38380
[junit4:junit4] 2> 653411 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 653415 T2071 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b165b74 name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 653415 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 653416 T2064 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 653447 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 653449 T2073 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77166a8b name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 653449 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 653450 T2064 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 653536 T2064 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 653564 T2064 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 653616 T2064 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 653640 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 653641 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 653702 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 653703 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 653811 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 653812 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 653816 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 653817 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 653828 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 653829 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 653832 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 653833 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 653836 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 653836 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 653839 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 653840 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 653843 T2064 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 653844 T2064 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 654121 T2064 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 654126 T2064 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15661
[junit4:junit4] 2> 654126 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 654127 T2064 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 654128 T2064 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606
[junit4:junit4] 2> 654128 T2064 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/solr.xml
[junit4:junit4] 2> 654129 T2064 oasc.CoreContainer.<init> New CoreContainer 1866368917
[junit4:junit4] 2> 654130 T2064 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/'
[junit4:junit4] 2> 654130 T2064 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/'
[junit4:junit4] 2> 654263 T2064 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 654264 T2064 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 654265 T2064 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 654265 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 654266 T2064 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 654267 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 654267 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 654268 T2064 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 654268 T2064 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 654269 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 654286 T2064 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 654287 T2064 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38380/solr
[junit4:junit4] 2> 654287 T2064 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 654289 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 654316 T2084 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@500382 name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 654317 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 654320 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 654332 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 654334 T2086 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c7e12ec name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 654334 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 654337 T2064 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 654348 T2064 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 654351 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 654354 T2064 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15661_
[junit4:junit4] 2> 654361 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15661_
[junit4:junit4] 2> 654364 T2064 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 654376 T2064 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 654385 T2064 oasc.Overseer.start Overseer (id=89945143710056451-127.0.0.1:15661_-n_0000000000) starting
[junit4:junit4] 2> 654390 T2064 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 654402 T2088 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 654403 T2064 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 654405 T2064 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 654413 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 654415 T2087 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 654419 T2089 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 654420 T2089 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 654421 T2089 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 655920 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 655920 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:15661_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:15661"}
[junit4:junit4] 2> 655921 T2087 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 655921 T2087 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 655936 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 656423 T2089 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 656423 T2089 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/collection1
[junit4:junit4] 2> 656423 T2089 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 656424 T2089 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 656424 T2089 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 656426 T2089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/collection1/'
[junit4:junit4] 2> 656428 T2089 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/collection1/lib/README' to classloader
[junit4:junit4] 2> 656428 T2089 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 656482 T2089 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 656543 T2089 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 656645 T2089 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 656651 T2089 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 657285 T2089 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 657297 T2089 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 657301 T2089 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 657315 T2089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 657320 T2089 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 657325 T2089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 657326 T2089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 657326 T2089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 657326 T2089 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 657327 T2089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 657328 T2089 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 657328 T2089 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 657328 T2089 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372453975606/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/control/data/
[junit4:junit4] 2> 657329 T2089 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9e6837
[junit4:junit4] 2> 657330 T2089 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/control/data
[junit4:junit4] 2> 657330 T2089 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/control/data/index/
[junit4:junit4] 2> 657330 T2089 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 657331 T2089 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/control/data/index
[junit4:junit4] 2> 657335 T2089 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_1,generation=1}
[junit4:junit4] 2> 657336 T2089 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 657339 T2089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 657339 T2089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 657340 T2089 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 657341 T2089 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 657341 T2089 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 657342 T2089 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 657342 T2089 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 657342 T2089 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 657343 T2089 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 657359 T2089 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 657367 T2089 oass.SolrIndexSearcher.<init> Opening Searcher@7bcf4920 main
[junit4:junit4] 2> 657368 T2089 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 657368 T2089 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 657372 T2090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7bcf4920 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 657374 T2089 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 657374 T2089 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15661 collection:control_collection shard:shard1
[junit4:junit4] 2> 657375 T2089 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 657395 T2089 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 657398 T2089 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 657398 T2089 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 657399 T2089 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15661/collection1/
[junit4:junit4] 2> 657399 T2089 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 657399 T2089 oasc.SyncStrategy.syncToMe http://127.0.0.1:15661/collection1/ has no replicas
[junit4:junit4] 2> 657399 T2089 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15661/collection1/ shard1
[junit4:junit4] 2> 657400 T2089 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 657440 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 657447 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 657457 T2089 oasc.ZkController.register We are http://127.0.0.1:15661/collection1/ and leader is http://127.0.0.1:15661/collection1/
[junit4:junit4] 2> 657457 T2089 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15661
[junit4:junit4] 2> 657457 T2089 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 657457 T2089 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 657458 T2089 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 657459 T2089 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 657460 T2064 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 657461 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 657462 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 657479 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 657480 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 657482 T2093 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ec9b name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 657483 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 657485 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 657487 T2064 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 657759 T2064 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 657762 T2064 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41296
[junit4:junit4] 2> 657763 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 657763 T2064 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 657764 T2064 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240
[junit4:junit4] 2> 657764 T2064 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/solr.xml
[junit4:junit4] 2> 657765 T2064 oasc.CoreContainer.<init> New CoreContainer 894381334
[junit4:junit4] 2> 657766 T2064 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/'
[junit4:junit4] 2> 657766 T2064 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/'
[junit4:junit4] 2> 657901 T2064 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 657902 T2064 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 657903 T2064 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 657903 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 657904 T2064 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 657904 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 657905 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 657905 T2064 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 657906 T2064 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 657907 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 657924 T2064 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 657925 T2064 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38380/solr
[junit4:junit4] 2> 657925 T2064 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 657927 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 657930 T2104 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2db0a817 name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 657930 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 657947 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 657959 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 657961 T2106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@757ae69c name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 657962 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 657974 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 658953 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 658955 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:15661_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:15661"}
[junit4:junit4] 2> 658980 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 658980 T2064 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41296_
[junit4:junit4] 2> 658980 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 658980 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 658984 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41296_
[junit4:junit4] 2> 658991 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 658991 T2086 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 658991 T2093 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 658991 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 658992 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 658992 T2106 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 658998 T2107 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 658999 T2107 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 659000 T2107 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 660486 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 660487 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41296_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41296"}
[junit4:junit4] 2> 660487 T2087 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 660488 T2087 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 660504 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 660504 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 660504 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 661002 T2107 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 661002 T2107 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/collection1
[junit4:junit4] 2> 661002 T2107 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 661003 T2107 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 661003 T2107 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 661005 T2107 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/collection1/'
[junit4:junit4] 2> 661006 T2107 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 661007 T2107 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/collection1/lib/README' to classloader
[junit4:junit4] 2> 661061 T2107 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 661123 T2107 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 661224 T2107 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 661230 T2107 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 661863 T2107 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 661876 T2107 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 661880 T2107 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 661894 T2107 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661899 T2107 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 661903 T2107 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661904 T2107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 661906 T2107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 661906 T2107 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 661907 T2107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 661907 T2107 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 661908 T2107 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 661908 T2107 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372453979240/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty1/
[junit4:junit4] 2> 661908 T2107 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9e6837
[junit4:junit4] 2> 661909 T2107 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty1
[junit4:junit4] 2> 661910 T2107 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty1/index/
[junit4:junit4] 2> 661910 T2107 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 661911 T2107 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty1/index
[junit4:junit4] 2> 661914 T2107 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 661915 T2107 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 661918 T2107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 661918 T2107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 661919 T2107 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 661920 T2107 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 661920 T2107 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 661921 T2107 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 661921 T2107 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 661922 T2107 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 661922 T2107 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 661937 T2107 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 661944 T2107 oass.SolrIndexSearcher.<init> Opening Searcher@37c44c8a main
[junit4:junit4] 2> 661945 T2107 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 661946 T2107 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 661950 T2108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37c44c8a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 661952 T2107 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 661952 T2107 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41296 collection:collection1 shard:shard1
[junit4:junit4] 2> 661953 T2107 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 661972 T2107 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 661975 T2107 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 661975 T2107 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 661975 T2107 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41296/collection1/
[junit4:junit4] 2> 661976 T2107 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 661976 T2107 oasc.SyncStrategy.syncToMe http://127.0.0.1:41296/collection1/ has no replicas
[junit4:junit4] 2> 661976 T2107 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41296/collection1/ shard1
[junit4:junit4] 2> 661976 T2107 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 662008 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 662015 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 662015 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 662015 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 662033 T2107 oasc.ZkController.register We are http://127.0.0.1:41296/collection1/ and leader is http://127.0.0.1:41296/collection1/
[junit4:junit4] 2> 662033 T2107 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41296
[junit4:junit4] 2> 662033 T2107 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 662033 T2107 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 662034 T2107 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 662035 T2107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 662036 T2064 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 662037 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 662038 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 662302 T2064 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 662305 T2064 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34359
[junit4:junit4] 2> 662306 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 662306 T2064 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 662307 T2064 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806
[junit4:junit4] 2> 662307 T2064 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/solr.xml
[junit4:junit4] 2> 662308 T2064 oasc.CoreContainer.<init> New CoreContainer 137585947
[junit4:junit4] 2> 662309 T2064 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/'
[junit4:junit4] 2> 662309 T2064 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/'
[junit4:junit4] 2> 662432 T2064 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 662433 T2064 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 662434 T2064 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 662434 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 662435 T2064 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 662435 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 662436 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 662436 T2064 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 662437 T2064 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 662437 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 662453 T2064 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 662454 T2064 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38380/solr
[junit4:junit4] 2> 662454 T2064 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 662455 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 662459 T2120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@200ffd11 name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 662460 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 662475 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 662487 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 662490 T2122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e669ca1 name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 662490 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 662499 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 663503 T2064 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34359_
[junit4:junit4] 2> 663538 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34359_
[junit4:junit4] 2> 663540 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 663580 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 663580 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41296_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41296"}
[junit4:junit4] 2> 663580 T2093 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 663580 T2122 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 663581 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 663609 T2106 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 663639 T2123 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 663639 T2123 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 663663 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 663663 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 663663 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 663664 T2123 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 663666 T2086 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 663667 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 663668 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 665168 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 665169 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34359_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34359"}
[junit4:junit4] 2> 665170 T2087 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 665170 T2087 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 665191 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 665191 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 665191 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 665191 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 665666 T2123 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 665666 T2123 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/collection1
[junit4:junit4] 2> 665666 T2123 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 665667 T2123 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 665667 T2123 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 665669 T2123 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/collection1/'
[junit4:junit4] 2> 665670 T2123 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/collection1/lib/README' to classloader
[junit4:junit4] 2> 665671 T2123 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 665725 T2123 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 665788 T2123 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 665890 T2123 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 665896 T2123 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 666522 T2123 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 666534 T2123 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 666538 T2123 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 666551 T2123 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 666556 T2123 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 666560 T2123 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 666561 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 666562 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 666563 T2123 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 666564 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 666564 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 666564 T2123 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 666565 T2123 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372453983806/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty2/
[junit4:junit4] 2> 666565 T2123 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9e6837
[junit4:junit4] 2> 666566 T2123 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty2
[junit4:junit4] 2> 666567 T2123 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty2/index/
[junit4:junit4] 2> 666567 T2123 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 666567 T2123 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty2/index
[junit4:junit4] 2> 666571 T2123 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e62d89f lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f0605),segFN=segments_1,generation=1}
[junit4:junit4] 2> 666571 T2123 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 666574 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 666575 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 666575 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 666576 T2123 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 666577 T2123 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 666577 T2123 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 666577 T2123 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 666578 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 666578 T2123 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 666594 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 666600 T2123 oass.SolrIndexSearcher.<init> Opening Searcher@4eb58470 main
[junit4:junit4] 2> 666602 T2123 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 666602 T2123 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 666606 T2124 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4eb58470 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 666609 T2123 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 666609 T2123 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34359 collection:collection1 shard:shard2
[junit4:junit4] 2> 666610 T2123 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 666627 T2123 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 666638 T2123 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 666638 T2123 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 666638 T2123 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34359/collection1/
[junit4:junit4] 2> 666639 T2123 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 666639 T2123 oasc.SyncStrategy.syncToMe http://127.0.0.1:34359/collection1/ has no replicas
[junit4:junit4] 2> 666639 T2123 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34359/collection1/ shard2
[junit4:junit4] 2> 666639 T2123 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 666695 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 666712 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 666712 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 666712 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 666712 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 666756 T2123 oasc.ZkController.register We are http://127.0.0.1:34359/collection1/ and leader is http://127.0.0.1:34359/collection1/
[junit4:junit4] 2> 666756 T2123 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34359
[junit4:junit4] 2> 666756 T2123 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 666757 T2123 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 666757 T2123 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 666759 T2123 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 666760 T2064 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 666761 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 666761 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 667024 T2064 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 667027 T2064 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:28517
[junit4:junit4] 2> 667028 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 667028 T2064 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 667029 T2064 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529
[junit4:junit4] 2> 667029 T2064 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/solr.xml
[junit4:junit4] 2> 667030 T2064 oasc.CoreContainer.<init> New CoreContainer 433437564
[junit4:junit4] 2> 667031 T2064 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/'
[junit4:junit4] 2> 667031 T2064 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/'
[junit4:junit4] 2> 667154 T2064 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 667155 T2064 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 667156 T2064 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 667156 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 667157 T2064 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 667157 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 667158 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 667158 T2064 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 667159 T2064 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 667159 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 667175 T2064 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 667176 T2064 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38380/solr
[junit4:junit4] 2> 667177 T2064 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 667178 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 667217 T2136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5da2a5db name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 667218 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 667228 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 667240 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 667242 T2138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35197b9a name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 667243 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 667255 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 668225 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 668226 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34359_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34359"}
[junit4:junit4] 2> 668242 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668242 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668242 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668242 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668242 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668258 T2064 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:28517_
[junit4:junit4] 2> 668260 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:28517_
[junit4:junit4] 2> 668263 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668263 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668263 T2086 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 668263 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 668264 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 668263 T2093 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 668265 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 668265 T2138 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 668265 T2106 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 668271 T2122 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 668276 T2139 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 668276 T2139 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 668278 T2139 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 669748 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 669749 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:28517_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:28517"}
[junit4:junit4] 2> 669750 T2087 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 669750 T2087 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 669770 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 669771 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 669771 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 669771 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 669770 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 670280 T2139 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 670280 T2139 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/collection1
[junit4:junit4] 2> 670280 T2139 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 670281 T2139 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 670281 T2139 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 670283 T2139 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/collection1/'
[junit4:junit4] 2> 670285 T2139 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 670285 T2139 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/collection1/lib/README' to classloader
[junit4:junit4] 2> 670339 T2139 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 670401 T2139 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 670502 T2139 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 670508 T2139 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 671139 T2139 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 671152 T2139 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 671156 T2139 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 671171 T2139 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 671176 T2139 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 671181 T2139 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 671182 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 671183 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 671183 T2139 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 671184 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 671185 T2139 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 671185 T2139 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 671185 T2139 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372453988529/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty3/
[junit4:junit4] 2> 671186 T2139 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9e6837
[junit4:junit4] 2> 671187 T2139 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty3
[junit4:junit4] 2> 671187 T2139 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty3/index/
[junit4:junit4] 2> 671187 T2139 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 671188 T2139 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty3/index
[junit4:junit4] 2> 671192 T2139 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 671192 T2139 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 671195 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 671196 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 671197 T2139 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 671197 T2139 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 671198 T2139 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 671198 T2139 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 671199 T2139 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 671199 T2139 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 671200 T2139 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 671216 T2139 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 671224 T2139 oass.SolrIndexSearcher.<init> Opening Searcher@2038bca main
[junit4:junit4] 2> 671225 T2139 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 671225 T2139 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 671229 T2140 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2038bca main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 671232 T2139 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 671232 T2139 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:28517 collection:collection1 shard:shard1
[junit4:junit4] 2> 671247 T2139 oasc.ZkController.register We are http://127.0.0.1:28517/collection1/ and leader is http://127.0.0.1:41296/collection1/
[junit4:junit4] 2> 671247 T2139 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:28517
[junit4:junit4] 2> 671248 T2139 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 671248 T2139 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C738 name=collection1 org.apache.solr.core.SolrCore@b0497b0 url=http://127.0.0.1:28517/collection1 node=127.0.0.1:28517_ C738_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:28517_, base_url=http://127.0.0.1:28517}
[junit4:junit4] 2> 671248 T2141 C738 P28517 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 671249 T2139 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 671249 T2141 C738 P28517 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 671249 T2141 C738 P28517 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 671250 T2141 C738 P28517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 671250 T2064 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 671251 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 671251 T2141 C738 P28517 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 671251 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 671275 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 671276 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:28517_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:28517"}
[junit4:junit4] 2> 671278 T2096 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 671288 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 671289 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 671289 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 671289 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 671288 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 671543 T2064 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 671546 T2064 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33550
[junit4:junit4] 2> 671547 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 671548 T2064 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 671548 T2064 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029
[junit4:junit4] 2> 671549 T2064 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/solr.xml
[junit4:junit4] 2> 671550 T2064 oasc.CoreContainer.<init> New CoreContainer 1353151821
[junit4:junit4] 2> 671550 T2064 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/'
[junit4:junit4] 2> 671551 T2064 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/'
[junit4:junit4] 2> 671683 T2064 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 671684 T2064 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 671685 T2064 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 671685 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 671686 T2064 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 671686 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 671687 T2064 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 671687 T2064 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 671688 T2064 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 671688 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 671705 T2064 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 671706 T2064 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38380/solr
[junit4:junit4] 2> 671707 T2064 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 671708 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 671710 T2153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15393f8e name:ZooKeeperConnection Watcher:127.0.0.1:38380 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 671711 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 671714 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 671727 T2064 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 671728 T2155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e8c6c1b name:ZooKeeperConnection Watcher:127.0.0.1:38380/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 671729 T2064 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 671741 T2064 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 672280 T2096 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 672280 T2096 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:28517_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 672744 T2064 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33550_
[junit4:junit4] 2> 672746 T2064 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33550_
[junit4:junit4] 2> 672749 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 672750 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 672749 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 672750 T2155 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672750 T2086 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672750 T2093 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672751 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672751 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672752 T2106 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672752 T2122 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672751 T2138 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 672766 T2156 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 672767 T2156 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 672768 T2156 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 672793 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 672794 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:33550_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33550"}
[junit4:junit4] 2> 672794 T2087 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 672794 T2087 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 672807 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672807 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672807 T2155 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672807 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672807 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 672807 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 673770 T2156 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 673770 T2156 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/collection1
[junit4:junit4] 2> 673771 T2156 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 673771 T2156 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 673772 T2156 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 673773 T2156 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/collection1/'
[junit4:junit4] 2> 673775 T2156 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 673776 T2156 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/collection1/lib/README' to classloader
[junit4:junit4] 2> 673829 T2156 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 673891 T2156 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 673993 T2156 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 673999 T2156 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C739 name=collection1 org.apache.solr.core.SolrCore@b0497b0 url=http://127.0.0.1:28517/collection1 node=127.0.0.1:28517_ C739_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:28517_, base_url=http://127.0.0.1:28517}
[junit4:junit4] 2> 674282 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41296/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 674282 T2141 C739 P28517 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:28517 START replicas=[http://127.0.0.1:41296/collection1/] nUpdates=100
[junit4:junit4] 2> 674283 T2141 C739 P28517 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 674284 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 674284 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 674284 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 674285 T2141 C739 P28517 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 674285 T2141 C739 P28517 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41296/collection1/. core=collection1
[junit4:junit4] 2> 674285 T2141 C739 P28517 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C740 name=collection1 org.apache.solr.core.SolrCore@63fc6493 url=http://127.0.0.1:41296/collection1 node=127.0.0.1:41296_ C740_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:41296_, base_url=http://127.0.0.1:41296, leader=true}
[junit4:junit4] 2> 674293 T2097 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 674296 T2098 C740 P41296 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 674299 T2098 C740 P41296 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 674299 T2098 C740 P41296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 674300 T2098 C740 P41296 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 674300 T2098 C740 P41296 oass.SolrIndexSearcher.<init> Opening Searcher@4112a0ca realtime
[junit4:junit4] 2> 674301 T2098 C740 P41296 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 674301 T2098 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 674302 T2141 C739 P28517 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 674303 T2141 C739 P28517 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 674305 T2099 C740 P41296 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 674306 T2099 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 674306 T2141 C739 P28517 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 674306 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 674307 T2141 C739 P28517 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 674307 T2141 C739 P28517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 674336 T2141 C739 P28517 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 674337 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 674338 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:28517_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:28517"}
[junit4:junit4] 2> 674358 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674358 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674358 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674358 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674358 T2155 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674358 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 674633 T2156 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 674645 T2156 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 674649 T2156 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 674663 T2156 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 674668 T2156 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 674672 T2156 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 674673 T2156 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 674674 T2156 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 674674 T2156 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 674675 T2156 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 674676 T2156 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 674676 T2156 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 674676 T2156 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372453993029/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty4/
[junit4:junit4] 2> 674677 T2156 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5e9e6837
[junit4:junit4] 2> 674678 T2156 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty4
[junit4:junit4] 2> 674678 T2156 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty4/index/
[junit4:junit4] 2> 674678 T2156 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 674679 T2156 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372453975060/jetty4/index
[junit4:junit4] 2> 674683 T2156 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57a83b1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cff4c5b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 674683 T2156 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 674686 T2156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 674687 T2156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 674688 T2156 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 674688 T2156 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 674689 T2156 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 674689 T2156 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 674690 T2156 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 674691 T2156 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 674691 T2156 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 674708 T2156 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 674715 T2156 oass.SolrIndexSearcher.<init> Opening Searcher@504c6e5c main
[junit4:junit4] 2> 674716 T2156 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 674717 T2156 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 674721 T2158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@504c6e5c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 674724 T2156 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 674724 T2156 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33550 collection:collection1 shard:shard2
[junit4:junit4] 2> 674727 T2156 oasc.ZkController.register We are http://127.0.0.1:33550/collection1/ and leader is http://127.0.0.1:34359/collection1/
[junit4:junit4] 2> 674727 T2156 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33550
[junit4:junit4] 2> 674727 T2156 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 674728 T2156 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C741 name=collection1 org.apache.solr.core.SolrCore@28f2e096 url=http://127.0.0.1:33550/collection1 node=127.0.0.1:33550_ C741_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:33550_, base_url=http://127.0.0.1:33550}
[junit4:junit4] 2> 674728 T2159 C741 P33550 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 674728 T2156 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 674728 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 674729 T2159 C741 P33550 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 674729 T2159 C741 P33550 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 674730 T2064 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4:junit4] 2> 674730 T2064 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 674731 T2159 C741 P33550 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 674731 T2064 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 674749 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 674750 T2112 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 674750 T2064 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 674751 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 675753 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 675863 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 675864 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:33550_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33550"}
[junit4:junit4] 2> 675880 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 675880 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 675880 T2155 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 675880 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 675880 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 675880 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 676752 T2112 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 676752 T2112 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:33550_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 676755 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 677757 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C741_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:33550_, base_url=http://127.0.0.1:33550}
[junit4:junit4] 2> 678754 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34359/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 678754 T2159 C741 P33550 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:33550 START replicas=[http://127.0.0.1:34359/collection1/] nUpdates=100
[junit4:junit4] 2> 678755 T2159 C741 P33550 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 678755 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 678755 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 678755 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 678756 T2159 C741 P33550 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 678757 T2159 C741 P33550 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34359/collection1/. core=collection1
[junit4:junit4] 2> 678757 T2159 C741 P33550 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 678759 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C742 name=collection1 org.apache.solr.core.SolrCore@11a9b86 url=http://127.0.0.1:34359/collection1 node=127.0.0.1:34359_ C742_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:34359_, base_url=http://127.0.0.1:34359, leader=true}
[junit4:junit4] 2> 678765 T2113 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 678770 T2114 C742 P34359 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 678773 T2114 C742 P34359 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e62d89f lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f0605),segFN=segments_1,generation=1}
[junit4:junit4] 2> 678773 T2114 C742 P34359 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 678774 T2114 C742 P34359 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 678774 T2114 C742 P34359 oass.SolrIndexSearcher.<init> Opening Searcher@ac8cbc9 realtime
[junit4:junit4] 2> 678775 T2114 C742 P34359 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 678775 T2114 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 678776 T2159 C741 P33550 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 678776 T2159 C741 P33550 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 678778 T2115 C742 P34359 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 678779 T2115 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 678780 T2159 C741 P33550 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 678780 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 678780 T2159 C741 P33550 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 678780 T2159 C741 P33550 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 678782 T2159 C741 P33550 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 678886 T2087 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 678887 T2087 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:33550_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33550"}
[junit4:junit4] 2> 678907 T2138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 678907 T2093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 678907 T2106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 678907 T2155 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 678907 T2086 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 678907 T2122 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 679761 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679763 T2064 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C743 name=collection1 org.apache.solr.core.SolrCore@1e6faf9d url=http://127.0.0.1:15661/collection1 node=127.0.0.1:15661_ C743_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:15661_, base_url=http://127.0.0.1:15661, leader=true}
[junit4:junit4] 2> 679774 T2076 C743 P15661 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_1,generation=1}
[junit4:junit4] 2> 679775 T2076 C743 P15661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 679777 T2076 C743 P15661 oass.SolrIndexSearcher.<init> Opening Searcher@3164c66b realtime
[junit4:junit4] 2> 679778 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439122327102029824)} 0 6
[junit4:junit4] 2> C741_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:33550_, base_url=http://127.0.0.1:33550}
[junit4:junit4] 2> 679806 T2145 C741 P33550 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@57a83b1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7cff4c5b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 679807 T2145 C741 P33550 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 679808 T2145 C741 P33550 oass.SolrIndexSearcher.<init> Opening Searcher@58da6bcb realtime
[junit4:junit4] 2> 679809 T2145 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439122327121952768&update.from=http://127.0.0.1:34359/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439122327121952768)} 0 5
[junit4:junit4] 2> C739_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:28517_, base_url=http://127.0.0.1:28517}
[junit4:junit4] 2> 679809 T2128 C739 P28517 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 679810 T2128 C739 P28517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 679812 T2128 C739 P28517 oass.SolrIndexSearcher.<init> Opening Searcher@7f2a6659 realtime
[junit4:junit4] 2> 679812 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439122327127195648&update.from=http://127.0.0.1:41296/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439122327127195648)} 0 5
[junit4:junit4] 2> 679813 T2100 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439122327127195648)} 0 17
[junit4:junit4] 2> 679814 T2116 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439122327121952768)} 0 23
[junit4:junit4] 2> 679815 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679817 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679820 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 679822 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679823 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679825 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679827 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679831 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679836 T2076 C743 P15661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_2,generation=2}
[junit4:junit4] 2> 679837 T2076 C743 P15661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 679838 T2076 C743 P15661 oass.SolrIndexSearcher.<init> Opening Searcher@36297ca9 main
[junit4:junit4] 2> 679839 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679840 T2090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36297ca9 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 679840 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439122327162847232)],commit=} 0 10
[junit4:junit4] 2> 679847 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439122327176478720)]} 0 1
[junit4:junit4] 2> 679848 T2101 C740 P41296 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41296/collection1/, StdNode: http://127.0.0.1:28517/collection1/, StdNode: http://127.0.0.1:34359/collection1/, StdNode: http://127.0.0.1:33550/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 679850 T2102 C740 P41296 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679850 T2147 C741 P33550 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679850 T2118 C742 P34359 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679851 T2128 C739 P28517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679851 T2147 C741 P33550 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 679852 T2118 C742 P34359 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 679853 T2147 C741 P33550 oass.SolrIndexSearcher.<init> Opening Searcher@5bb75053 main
[junit4:junit4] 2> 679853 T2118 C742 P34359 oass.SolrIndexSearcher.<init> Opening Searcher@6c36e470 main
[junit4:junit4] 2> 679854 T2147 C741 P33550 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679855 T2158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5bb75053 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 679855 T2118 C742 P34359 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679855 T2102 C740 P41296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_2,generation=2}
[junit4:junit4] 2> 679856 T2147 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 679856 T2128 C739 P28517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_2,generation=2}
[junit4:junit4] 2> 679858 T2128 C739 P28517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 679856 T2124 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c36e470 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 679857 T2102 C740 P41296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 679859 T2118 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4] 2> 679860 T2128 C739 P28517 oass.SolrIndexSearcher.<init> Opening Searcher@2c3b748d main
[junit4:junit4] 2> 679860 T2128 C739 P28517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679860 T2102 C740 P41296 oass.SolrIndexSearcher.<init> Opening Searcher@43adb5e6 main
[junit4:junit4] 2> 679861 T2140 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c3b748d main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 679862 T2102 C740 P41296 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679862 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 679862 T2108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43adb5e6 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 679863 T2102 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 679864 T2101 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439122327176478720)],commit=} 0 21
[junit4:junit4] 2> 679865 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679867 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679871 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 679874 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679876 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679879 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679882 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679883 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679885 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679891 T2112 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001640&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679891 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001640&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679894 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001640&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 679895 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 679901 T2114 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001650&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679901 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001650&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679904 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001650&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 679905 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 679906 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679913 T2175 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001662&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679913 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001662&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679917 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001662&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 679918 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 679924 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001673&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679924 T2179 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001673&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679927 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001673&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 679928 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 679933 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679937 T2076 C743 P15661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_3,generation=3}
[junit4:junit4] 2> 679938 T2076 C743 P15661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 679939 T2076 C743 P15661 oass.SolrIndexSearcher.<init> Opening Searcher@613efbf8 main
[junit4:junit4] 2> 679940 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679941 T2090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@613efbf8 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 679941 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439122327268753408)],commit=} 0 10
[junit4:junit4] 2> 679954 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439122327286579200)]} 0 2
[junit4:junit4] 2> 679955 T2100 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34359/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439122327286579200)]} 0 8
[junit4:junit4] 2> 679956 T2117 C742 P34359 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41296/collection1/, StdNode: http://127.0.0.1:28517/collection1/, StdNode: http://127.0.0.1:34359/collection1/, StdNode: http://127.0.0.1:33550/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 679958 T2130 C739 P28517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679958 T2100 C740 P41296 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679958 T2182 C742 P34359 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679960 T2145 C741 P33550 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 679961 T2182 C742 P34359 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 679962 T2145 C741 P33550 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 679962 T2182 C742 P34359 oass.SolrIndexSearcher.<init> Opening Searcher@9fe774 main
[junit4:junit4] 2> 679963 T2145 C741 P33550 oass.SolrIndexSearcher.<init> Opening Searcher@c0634c3 main
[junit4:junit4] 2> 679963 T2182 C742 P34359 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679964 T2145 C741 P33550 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679965 T2158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c0634c3 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 679965 T2124 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9fe774 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 679965 T2100 C740 P41296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_3,generation=3}
[junit4:junit4] 2> 679965 T2130 C739 P28517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_3,generation=3}
[junit4:junit4] 2> 679967 T2100 C740 P41296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 679966 T2182 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 679966 T2145 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 679967 T2130 C739 P28517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 679969 T2100 C740 P41296 oass.SolrIndexSearcher.<init> Opening Searcher@54f47777 main
[junit4:junit4] 2> 679970 T2100 C740 P41296 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679970 T2130 C739 P28517 oass.SolrIndexSearcher.<init> Opening Searcher@10e0fee0 main
[junit4:junit4] 2> 679970 T2108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54f47777 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 679971 T2130 C739 P28517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 679972 T2100 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 679972 T2140 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10e0fee0 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 679973 T2130 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 679973 T2117 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 679975 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679976 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679979 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679982 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 679983 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679987 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679990 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679991 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679993 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 679998 T2148 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001748&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 679998 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001748&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680001 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001748&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680002 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 680007 T2149 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001757&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 680007 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001757&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680010 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001757&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680011 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 680012 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680017 T2150 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001767&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 680017 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001767&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680020 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001767&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680021 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 680025 T2151 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001776&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680026 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001776&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680029 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001776&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680030 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=6
[junit4:junit4] 2> 680035 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680039 T2076 C743 P15661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_4,generation=4}
[junit4:junit4] 2> 680040 T2076 C743 P15661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 680041 T2076 C743 P15661 oass.SolrIndexSearcher.<init> Opening Searcher@893074b main
[junit4:junit4] 2> 680042 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680042 T2090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@893074b main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 680043 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439122327375708160)],commit=} 0 10
[junit4:junit4] 2> 680055 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439122327392485376)]} 0 2
[junit4:junit4] 2> 680056 T2184 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:28517/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439122327392485376)]} 0 7
[junit4:junit4] 2> 680056 T2129 C739 P28517 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41296/collection1/, StdNode: http://127.0.0.1:28517/collection1/, StdNode: http://127.0.0.1:34359/collection1/, StdNode: http://127.0.0.1:33550/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 680059 T2131 C739 P28517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680059 T2189 C741 P33550 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680059 T2184 C740 P41296 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680059 T2188 C742 P34359 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680060 T2189 C741 P33550 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 680061 T2188 C742 P34359 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 680061 T2189 C741 P33550 oass.SolrIndexSearcher.<init> Opening Searcher@439b5938 main
[junit4:junit4] 2> 680062 T2188 C742 P34359 oass.SolrIndexSearcher.<init> Opening Searcher@6174fe1a main
[junit4:junit4] 2> 680062 T2189 C741 P33550 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680063 T2158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@439b5938 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 680063 T2131 C739 P28517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_4,generation=4}
[junit4:junit4] 2> 680063 T2188 C742 P34359 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680064 T2131 C739 P28517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 680064 T2189 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 680064 T2184 C740 P41296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_4,generation=4}
[junit4:junit4] 2> 680064 T2124 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6174fe1a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 680066 T2131 C739 P28517 oass.SolrIndexSearcher.<init> Opening Searcher@3fe6a999 main
[junit4:junit4] 2> 680066 T2184 C740 P41296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 680067 T2131 C739 P28517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680067 T2188 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 8
[junit4:junit4] 2> 680068 T2140 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3fe6a999 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 680069 T2184 C740 P41296 oass.SolrIndexSearcher.<init> Opening Searcher@3e641550 main
[junit4:junit4] 2> 680069 T2131 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4] 2> 680069 T2184 C740 P41296 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680070 T2108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e641550 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 680071 T2184 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4] 2> 680071 T2129 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 25
[junit4:junit4] 2> 680073 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680074 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680078 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 680080 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680082 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680085 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680088 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680089 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680091 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680096 T2112 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001846&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680096 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001846&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680100 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001846&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680101 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680106 T2114 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001856&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680106 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001856&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680110 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001856&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680111 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680112 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680117 T2175 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001867&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680117 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001867&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680121 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001867&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680122 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680127 T2179 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001877&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680127 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001877&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680131 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001877&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680132 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680137 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680141 T2076 C743 P15661 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3324f454 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d2ca658),segFN=segments_5,generation=5}
[junit4:junit4] 2> 680142 T2076 C743 P15661 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 680143 T2076 C743 P15661 oass.SolrIndexSearcher.<init> Opening Searcher@375c787f main
[junit4:junit4] 2> 680144 T2076 C743 P15661 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680145 T2090 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@375c787f main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 680145 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439122327482662912)],commit=} 0 10
[junit4:junit4] 2> 680157 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439122327499440128)]} 0 2
[junit4:junit4] 2> 680158 T2191 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:33550/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439122327499440128)]} 0 7
[junit4:junit4] 2> 680159 T2146 C741 P33550 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:41296/collection1/, StdNode: http://127.0.0.1:28517/collection1/, StdNode: http://127.0.0.1:34359/collection1/, StdNode: http://127.0.0.1:33550/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 680162 T2132 C739 P28517 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680162 T2195 C742 P34359 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680162 T2196 C741 P33550 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680162 T2191 C740 P41296 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 680163 T2196 C741 P33550 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 680163 T2195 C742 P34359 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 680164 T2196 C741 P33550 oass.SolrIndexSearcher.<init> Opening Searcher@3971ad51 main
[junit4:junit4] 2> 680165 T2195 C742 P34359 oass.SolrIndexSearcher.<init> Opening Searcher@6455f9ac main
[junit4:junit4] 2> 680165 T2196 C741 P33550 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680166 T2158 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3971ad51 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 680166 T2195 C742 P34359 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680166 T2132 C739 P28517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@554bf9ba lockFactory=org.apache.lucene.store.NativeFSLockFactory@5d9907c3),segFN=segments_5,generation=5}
[junit4:junit4] 2> 680167 T2191 C740 P41296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@102a5193 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ef9d91e),segFN=segments_5,generation=5}
[junit4:junit4] 2> 680167 T2196 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 680167 T2124 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6455f9ac main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 680168 T2191 C740 P41296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 680168 T2132 C739 P28517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 680169 T2195 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 680171 T2191 C740 P41296 oass.SolrIndexSearcher.<init> Opening Searcher@534b648d main
[junit4:junit4] 2> 680171 T2132 C739 P28517 oass.SolrIndexSearcher.<init> Opening Searcher@2c9d1b58 main
[junit4:junit4] 2> 680172 T2191 C740 P41296 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680173 T2108 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@534b648d main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 680173 T2132 C739 P28517 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 680173 T2191 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 680173 T2140 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c9d1b58 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 680175 T2132 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 680175 T2146 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 27
[junit4:junit4] 2> 680176 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680178 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680182 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 680184 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680186 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680189 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680192 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680193 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680195 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680200 T2148 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001950&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680200 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001950&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680204 T2096 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001950&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680205 T2101 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680210 T2149 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001960&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680210 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001960&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680214 T2098 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001960&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680215 T2129 C739 P28517 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680216 T2064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 680221 T2150 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001971&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680221 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001971&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680225 T2174 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001971&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680226 T2117 C742 P34359 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680231 T2151 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001981&shard.url=127.0.0.1:34359/collection1/|127.0.0.1:33550/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 680231 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001981&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 680235 T2178 C740 P41296 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372454001981&shard.url=127.0.0.1:41296/collection1/|127.0.0.1:28517/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 680236 T2146 C741 P33550 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 680242 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439122327591714816)]} 0 3
[junit4:junit4] 2> 680252 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439122327596957696)]} 0 3
[junit4:junit4] 2> 680253 T2197 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1439122327596957696)]} 0 9
[junit4:junit4] 2> 680256 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1439122327608492032)]} 0 1
[junit4:junit4] 2> 680266 T2145 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34359/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1439122327612686336)]} 0 2
[junit4:junit4] 2> 680267 T2116 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1439122327612686336)]} 0 8
[junit4:junit4] 2> 680271 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1439122327624220672)]} 0 1
[junit4:junit4] 2> 680278 T2145 C741 P33550 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:34359/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1439122327628414976)]} 0 1
[junit4:junit4] 2> 680279 T2116 C742 P34359 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1439122327628414976)]} 0 5
[junit4:junit4] 2> 680283 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1439122327636803584)]} 0 1
[junit4:junit4] 2> 680291 T2128 C739 P28517 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:41296/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1439122327640997888)]} 0 1
[junit4:junit4] 2> 680292 T2197 C740 P41296 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1439122327640997888)]} 0 6
[junit4:junit4] 2> 680296 T2076 C743 P15661 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1439122327650435072)]} 0 1
[junit4:junit4] 2> 680307 T2145 C741 P33550 oasup.LogUpdate
[...truncated too long message...]
ield_s=Lucene41(blocksize=128), sequence_i=Lucene41(blocksize=128), name=Lucene41(blocksize=128), foo_i=PostingsFormat(name=NestedPulsing), regex_dup_B_s=PostingsFormat(name=Memory doPackFST= true), multiDefault=PostingsFormat(name=Memory doPackFST= true), n_tf1=PostingsFormat(name=Memory doPackFST= true), n_dt1=PostingsFormat(name=NestedPulsing), genre_s=Lucene41(blocksize=128), author_t=PostingsFormat(name=NestedPulsing), range_facet_l=PostingsFormat(name=Memory doPackFST= true), n_ti1=Lucene41(blocksize=128), text=Pulsing41(freqCutoff=1 minBlockSize=24 maxBlockSize=73), _version_=PostingsFormat(name=NestedPulsing), val_i=Lucene41(blocksize=128), SubjectTerms_mfacet=PostingsFormat(name=NestedPulsing), series_t=PostingsFormat(name=NestedPulsing), a_t=PostingsFormat(name=Memory doPackFST= true), n_tdt1=PostingsFormat(name=NestedPulsing), regex_dup_A_s=Pulsing41(freqCutoff=1 minBlockSize=24 maxBlockSize=73), price=Pulsing41(freqCutoff=1 minBlockSize=24 maxBlockSize=73), other_tl1=PostingsFormat(name=NestedPulsing), n_l1=PostingsFormat(name=Memory doPackFST= true), a_si=PostingsFormat(name=NestedPulsing), inStock=Pulsing41(freqCutoff=1 minBlockSize=24 maxBlockSize=73)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=nl, timezone=Australia/Yancowinna
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=129218872,total=524288000
[junit4:junit4] 2> NOTE: All tests run in this JVM: [HdfsDirectoryTest, TestIndexSearcher, TestUtils, ShardRoutingCustomTest, MultiTermTest, TestCSVResponseWriter, UpdateParamsTest, SuggesterTSTTest, TestRTGBase, TestDFRSimilarityFactory, TestNumberUtils, TestArbitraryIndexDir, TestRangeQuery, TestFiltering, TestZkChroot, TestReload, JSONWriterTest, TestLazyCores, SimplePostToolTest, TestUpdate, TestFieldCollectionResource, TestSchemaSimilarityResource, TimeZoneUtilsTest, NotRequiredUniqueKeyTest, TestDistributedSearch, OutputWriterTest, TestClassNameShortening, SignatureUpdateProcessorFactoryTest, TestSolrQueryParserResource, DirectSolrSpellCheckerTest, PathHierarchyTokenizerFactoryTest, SampleTest, TestMultiCoreConfBootstrap, TestDefaultSimilarityFactory, XmlUpdateRequestHandlerTest, FileUtilsTest, TestFastOutputStream, SpellCheckCollatorTest, TestJmxMonitoredMap, HdfsUnloadDistributedZkTest, TestSolrXMLSerializer, TestLMJelinekMercerSimilarityFactory, BadIndexSchemaTest, CollectionsAPIDistributedZkTest, CurrencyFieldXmlFileTest, TestUniqueKeyFieldResource, TestDistributedGrouping, TestRandomFaceting, TestSuggestSpellingConverter, TestReloadAndDeleteDocs, PingRequestHandlerTest, DOMUtilTest, IndexSchemaRuntimeFieldTest, ConvertedLegacyTest, RequestHandlersTest, FullSolrCloudDistribCmdsTest, IndexReaderFactoryTest, TestCoreDiscovery, FieldAnalysisRequestHandlerTest, TestRecoveryHdfs, HighlighterConfigTest, TestStressReorder, DocValuesMultiTest, TestLuceneMatchVersion, DirectUpdateHandlerTest, CurrencyFieldOpenExchangeTest, TestWordDelimiterFilterFactory, SuggesterWFSTTest, ZkSolrClientTest, CachingDirectoryFactoryTest, QueryResultKeyTest, DateMathParserTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=7423CCDB0A7E2DF7 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl -Dtests.timezone=Australia/Yancowinna -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 0.00s J1 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([7423CCDB0A7E2DF7]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=2550, name=recoveryCmdExecutor-1338-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([7423CCDB0A7E2DF7]:0)
[junit4:junit4] Completed on J1 in 236.95s, 1 test, 2 errors <<< FAILURES!
[...truncated 493 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 631 ignored
Total time: 41 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4095 - Still Failing
Posted by Mark Miller <ma...@gmail.com>.
Strange that we are seeing this thread get stuck on freebsd - it has a connect timeout.
- Mark
On Jun 28, 2013, at 9:10 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4095/
>
> 2 tests failed.
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
>
>
> FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
>
> Error Message:
> There are still zombie threads that couldn't be terminated: 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> at java.net.Socket.connect(Socket.java:579)
> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:722)
> at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
>
>
>
>
> Build Log:
> [...truncated 9329 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4] 2> 351088 T596 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
> [junit4:junit4] 2> 351096 T596 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372424293864
> [junit4:junit4] 2> 351097 T596 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4] 2> 351098 T597 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4] 2> 351199 T596 oasc.ZkTestServer.run start zk server on port:36823
> [junit4:junit4] 2> 351200 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 351206 T603 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@215dda5b name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 351207 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 351207 T596 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4] 2> 351212 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 351221 T605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@287cbb41 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 351221 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 351222 T596 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4] 2> 351225 T596 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4] 2> 351234 T596 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4] 2> 351237 T596 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4] 2> 351240 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 351241 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 351247 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4] 2> 351248 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4] 2> 351353 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 351354 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 351358 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4] 2> 351359 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4] 2> 351368 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4] 2> 351369 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4] 2> 351373 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 351374 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 351378 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 351378 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 351382 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 351383 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 351386 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 351387 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 351662 T596 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 351666 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14091
> [junit4:junit4] 2> 351667 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 351667 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 351668 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158
> [junit4:junit4] 2> 351669 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/solr.xml
> [junit4:junit4] 2> 351669 T596 oasc.CoreContainer.<init> New CoreContainer 52426110
> [junit4:junit4] 2> 351670 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/'
> [junit4:junit4] 2> 351670 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/'
> [junit4:junit4] 2> 351817 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 351818 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 351819 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 351819 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 351820 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 351820 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 351821 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 351821 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 351822 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 351822 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 351842 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 351843 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
> [junit4:junit4] 2> 351843 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 351844 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 351848 T616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bc90bce name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 351848 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 351858 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 351872 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 351874 T618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43ec9e1e name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 351875 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 351882 T596 oascc.SolrZkClient.makePath makePath: /overseer/queue
> [junit4:junit4] 2> 351888 T596 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
> [junit4:junit4] 2> 351893 T596 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4] 2> 351895 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14091_
> [junit4:junit4] 2> 351902 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14091_
> [junit4:junit4] 2> 351907 T596 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4] 2> 351924 T596 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 351928 T596 oasc.Overseer.start Overseer (id=89943198522998787-127.0.0.1:14091_-n_0000000000) starting
> [junit4:junit4] 2> 351940 T596 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
> [junit4:junit4] 2> 351953 T620 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 351954 T596 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4] 2> 351957 T596 oascc.SolrZkClient.makePath makePath: /aliases.json
> [junit4:junit4] 2> 351960 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 351964 T619 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 351968 T621 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 351968 T621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 351970 T621 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 353469 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 353470 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:14091_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:14091"}
> [junit4:junit4] 2> 353471 T619 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4] 2> 353471 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 353486 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 353972 T621 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 353972 T621 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1
> [junit4:junit4] 2> 353972 T621 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4] 2> 353973 T621 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 353973 T621 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4] 2> 353975 T621 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/'
> [junit4:junit4] 2> 353977 T621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 353978 T621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/lib/README' to classloader
> [junit4:junit4] 2> 354031 T621 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 354093 T621 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 354196 T621 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 354203 T621 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 354863 T621 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 354876 T621 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 354880 T621 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 354900 T621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 354905 T621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 354910 T621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 354911 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 354912 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 354912 T621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 354913 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 354914 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 354914 T621 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 354915 T621 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/
> [junit4:junit4] 2> 354915 T621 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
> [junit4:junit4] 2> 354916 T621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data
> [junit4:junit4] 2> 354916 T621 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index/
> [junit4:junit4] 2> 354917 T621 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 354917 T621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index
> [junit4:junit4] 2> 354922 T621 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 354922 T621 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 354925 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 354925 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 354926 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 354928 T621 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 354928 T621 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 354929 T621 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 354944 T621 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 354951 T621 oass.SolrIndexSearcher.<init> Opening Searcher@14639129 main
> [junit4:junit4] 2> 354952 T621 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 354953 T621 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 354957 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14639129 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 354959 T621 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 354959 T621 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14091 collection:control_collection shard:shard1
> [junit4:junit4] 2> 354960 T621 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4] 2> 354981 T621 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 354993 T621 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 354993 T621 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:14091/collection1/
> [junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.syncToMe http://127.0.0.1:14091/collection1/ has no replicas
> [junit4:junit4] 2> 354995 T621 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:14091/collection1/ shard1
> [junit4:junit4] 2> 354995 T621 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4] 2> 356496 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 356525 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 356542 T621 oasc.ZkController.register We are http://127.0.0.1:14091/collection1/ and leader is http://127.0.0.1:14091/collection1/
> [junit4:junit4] 2> 356542 T621 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14091
> [junit4:junit4] 2> 356542 T621 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 356543 T621 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 356543 T621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 356545 T621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 356546 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 356547 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 356548 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 356567 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4] 2> 356569 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 356571 T625 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f14acc4 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 356572 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 356574 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 356577 T596 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4] 2> 356848 T596 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 356851 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24273
> [junit4:junit4] 2> 356851 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 356852 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 356853 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344
> [junit4:junit4] 2> 356853 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/solr.xml
> [junit4:junit4] 2> 356854 T596 oasc.CoreContainer.<init> New CoreContainer 1036840427
> [junit4:junit4] 2> 356854 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/'
> [junit4:junit4] 2> 356855 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/'
> [junit4:junit4] 2> 356989 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 356990 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 356991 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 356991 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 356992 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 356993 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 356993 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 356994 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 356994 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 356995 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 357012 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 357013 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
> [junit4:junit4] 2> 357014 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 357015 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 357019 T636 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eb79d53 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 357019 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 357033 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 357045 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 357048 T638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d692086 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 357048 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 357060 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 358041 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 358042 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:14091_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:14091"}
> [junit4:junit4] 2> 358059 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 358059 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 358059 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 358064 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24273_
> [junit4:junit4] 2> 358066 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24273_
> [junit4:junit4] 2> 358073 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 358074 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 358074 T638 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 358074 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 358076 T625 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 358076 T618 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 358083 T639 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 358083 T639 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 358085 T639 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 359566 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 359567 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:24273_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:24273"}
> [junit4:junit4] 2> 359568 T619 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4] 2> 359568 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 359582 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 359582 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 359582 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 360087 T639 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 360087 T639 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1
> [junit4:junit4] 2> 360087 T639 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 360088 T639 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 360088 T639 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 360090 T639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/'
> [junit4:junit4] 2> 360092 T639 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/lib/README' to classloader
> [junit4:junit4] 2> 360093 T639 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 360158 T639 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 360220 T639 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 360323 T639 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 360332 T639 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 361000 T639 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 361014 T639 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 361018 T639 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 361039 T639 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 361044 T639 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 361049 T639 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 361051 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 361051 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 361052 T639 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 361053 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 361053 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 361054 T639 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 361054 T639 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/
> [junit4:junit4] 2> 361055 T639 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
> [junit4:junit4] 2> 361056 T639 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1
> [junit4:junit4] 2> 361056 T639 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index/
> [junit4:junit4] 2> 361056 T639 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 361057 T639 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index
> [junit4:junit4] 2> 361061 T639 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 361061 T639 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 361065 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 361065 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 361066 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 361067 T639 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 361069 T639 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 361070 T639 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 361086 T639 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 361094 T639 oass.SolrIndexSearcher.<init> Opening Searcher@274be5a4 main
> [junit4:junit4] 2> 361095 T639 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 361095 T639 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 361101 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@274be5a4 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 361104 T639 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 361104 T639 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24273 collection:collection1 shard:shard1
> [junit4:junit4] 2> 361105 T639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4] 2> 361132 T639 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
> [junit4:junit4] 2> 361137 T639 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 361137 T639 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 361137 T639 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24273/collection1/
> [junit4:junit4] 2> 361138 T639 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 361138 T639 oasc.SyncStrategy.syncToMe http://127.0.0.1:24273/collection1/ has no replicas
> [junit4:junit4] 2> 361138 T639 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24273/collection1/ shard1
> [junit4:junit4] 2> 361139 T639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 362590 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 362621 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 362621 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 362621 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 362635 T639 oasc.ZkController.register We are http://127.0.0.1:24273/collection1/ and leader is http://127.0.0.1:24273/collection1/
> [junit4:junit4] 2> 362635 T639 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24273
> [junit4:junit4] 2> 362635 T639 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 362636 T639 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 362636 T639 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 362638 T639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 362639 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 362640 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 362641 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 362975 T596 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 362979 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17310
> [junit4:junit4] 2> 362981 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 362981 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 362982 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430
> [junit4:junit4] 2> 362982 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/solr.xml
> [junit4:junit4] 2> 362983 T596 oasc.CoreContainer.<init> New CoreContainer 14294857
> [junit4:junit4] 2> 362984 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/'
> [junit4:junit4] 2> 362984 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/'
> [junit4:junit4] 2> 363136 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 363137 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 363138 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 363138 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 363139 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 363139 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 363140 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 363140 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 363141 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 363142 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 363160 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 363161 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
> [junit4:junit4] 2> 363162 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 363163 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 363166 T652 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51e23255 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 363167 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 363177 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 363190 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 363192 T654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fce68d name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 363193 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 363203 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 364129 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 364130 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node1",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:24273_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:24273"}
> [junit4:junit4] 2> 364148 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364148 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364148 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364148 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364207 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17310_
> [junit4:junit4] 2> 364210 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17310_
> [junit4:junit4] 2> 364212 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364213 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364212 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 364213 T638 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 364214 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 364215 T625 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 364215 T618 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 364216 T654 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 364222 T655 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 364222 T655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 364224 T655 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 365656 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 365657 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:17310_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:17310"}
> [junit4:junit4] 2> 365658 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 365658 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 365685 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 365685 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 365685 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 365685 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 366226 T655 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 366226 T655 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1
> [junit4:junit4] 2> 366226 T655 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 366227 T655 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 366227 T655 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 366229 T655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/'
> [junit4:junit4] 2> 366231 T655 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/lib/README' to classloader
> [junit4:junit4] 2> 366232 T655 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 366298 T655 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 366362 T655 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 366464 T655 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 366472 T655 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 367149 T655 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 367165 T655 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 367169 T655 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 367190 T655 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 367196 T655 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 367201 T655 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 367203 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 367203 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 367204 T655 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 367205 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 367205 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 367206 T655 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 367206 T655 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/
> [junit4:junit4] 2> 367207 T655 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
> [junit4:junit4] 2> 367208 T655 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2
> [junit4:junit4] 2> 367208 T655 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index/
> [junit4:junit4] 2> 367208 T655 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 367209 T655 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index
> [junit4:junit4] 2> 367213 T655 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@555673be lockFactory=org.apache.lucene.store.NativeFSLockFactory@29590fb3),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 367214 T655 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 367217 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 367218 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 367218 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 367219 T655 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 367220 T655 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 367220 T655 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 367221 T655 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 367222 T655 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 367222 T655 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 367242 T655 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 367251 T655 oass.SolrIndexSearcher.<init> Opening Searcher@50776c5f main
> [junit4:junit4] 2> 367252 T655 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 367252 T655 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 367258 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50776c5f main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 367261 T655 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 367261 T655 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:17310 collection:collection1 shard:shard2
> [junit4:junit4] 2> 367262 T655 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4] 2> 367302 T655 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
> [junit4:junit4] 2> 367313 T655 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 367313 T655 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:17310/collection1/
> [junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.syncToMe http://127.0.0.1:17310/collection1/ has no replicas
> [junit4:junit4] 2> 367315 T655 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:17310/collection1/ shard2
> [junit4:junit4] 2> 367315 T655 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 368693 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 368712 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 368712 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 368712 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 368712 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 368759 T655 oasc.ZkController.register We are http://127.0.0.1:17310/collection1/ and leader is http://127.0.0.1:17310/collection1/
> [junit4:junit4] 2> 368759 T655 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:17310
> [junit4:junit4] 2> 368759 T655 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 368759 T655 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 368759 T655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 368761 T655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 368763 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 368763 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 368764 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 369048 T596 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 369052 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35832
> [junit4:junit4] 2> 369053 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 369053 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 369054 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550
> [junit4:junit4] 2> 369054 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/solr.xml
> [junit4:junit4] 2> 369055 T596 oasc.CoreContainer.<init> New CoreContainer 2003196108
> [junit4:junit4] 2> 369056 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/'
> [junit4:junit4] 2> 369056 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/'
> [junit4:junit4] 2> 369188 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 369189 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 369190 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 369190 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 369191 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 369191 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 369192 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 369192 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 369193 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 369193 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 369212 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 369213 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
> [junit4:junit4] 2> 369214 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 369215 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 369218 T668 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d6e8b68 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 369219 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 369236 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 369247 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 369250 T670 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4aca485d name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 369250 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 369261 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 370226 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 370227 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node2",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:17310_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:17310"}
> [junit4:junit4] 2> 370241 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370241 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370241 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370241 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370241 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370265 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35832_
> [junit4:junit4] 2> 370267 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35832_
> [junit4:junit4] 2> 370270 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370270 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370270 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 370271 T670 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 370271 T638 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 370272 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 370272 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 370273 T625 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 370273 T618 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 370274 T654 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 370280 T671 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 370280 T671 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 370282 T671 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 371748 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 371749 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
> [junit4:junit4] 2> 371750 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 371750 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 371780 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 371781 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 371781 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 371781 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 371780 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 372284 T671 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 372284 T671 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1
> [junit4:junit4] 2> 372284 T671 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 372285 T671 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 372286 T671 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 372288 T671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/'
> [junit4:junit4] 2> 372289 T671 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/lib/README' to classloader
> [junit4:junit4] 2> 372290 T671 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 372355 T671 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 372418 T671 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 372520 T671 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 372527 T671 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> 373195 T671 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 373210 T671 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 373214 T671 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 373234 T671 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 373240 T671 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 373245 T671 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 373246 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 373247 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 373247 T671 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 373249 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 373249 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 373249 T671 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 373250 T671 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/
> [junit4:junit4] 2> 373250 T671 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
> [junit4:junit4] 2> 373251 T671 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3
> [junit4:junit4] 2> 373252 T671 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index/
> [junit4:junit4] 2> 373252 T671 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 373252 T671 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index
> [junit4:junit4] 2> 373256 T671 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 373257 T671 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 373260 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 373261 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 373262 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 373263 T671 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 373263 T671 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 373264 T671 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 373264 T671 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 373265 T671 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 373265 T671 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 373283 T671 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 373292 T671 oass.SolrIndexSearcher.<init> Opening Searcher@40962cea main
> [junit4:junit4] 2> 373293 T671 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 373294 T671 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 373299 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40962cea main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 373302 T671 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 373303 T671 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35832 collection:collection1 shard:shard1
> [junit4:junit4] 2> 373313 T671 oasc.ZkController.register We are http://127.0.0.1:35832/collection1/ and leader is http://127.0.0.1:24273/collection1/
> [junit4:junit4] 2> 373313 T671 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35832
> [junit4:junit4] 2> 373314 T671 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 373314 T671 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C624 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C624_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
> [junit4:junit4] 2> 373314 T673 C624 P35832 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 373315 T673 C624 P35832 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 373315 T671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 373315 T673 C624 P35832 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 373316 T673 C624 P35832 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 373317 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 373317 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 373318 T673 C624 P35832 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 373318 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 373342 T628 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 373643 T596 oejs.Server.doStart jetty-8.1.10.v20130312
> [junit4:junit4] 2> 373646 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:20834
> [junit4:junit4] 2> 373647 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 373648 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 373648 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108
> [junit4:junit4] 2> 373649 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/solr.xml
> [junit4:junit4] 2> 373649 T596 oasc.CoreContainer.<init> New CoreContainer 1561369995
> [junit4:junit4] 2> 373650 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/'
> [junit4:junit4] 2> 373651 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/'
> [junit4:junit4] 2> 373798 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 373799 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 373799 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 373800 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 373800 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 373801 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 373801 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 373802 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 373802 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 373803 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 373822 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
> [junit4:junit4] 2> 373822 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
> [junit4:junit4] 2> 373823 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 373824 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 373827 T685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f7cd760 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 373828 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 373844 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 373857 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 373859 T687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3df988fd name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 373860 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 373866 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 374795 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 374796 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
> [junit4:junit4] 2> 374809 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374810 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374810 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374810 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374810 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374810 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374870 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:20834_
> [junit4:junit4] 2> 374872 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:20834_
> [junit4:junit4] 2> 374875 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374875 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374875 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374876 T670 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374875 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 374876 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 374876 T638 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374877 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 374877 T687 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374878 T618 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374878 T654 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374879 T625 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4] 2> 374885 T688 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 374885 T688 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 374887 T688 oasc.ZkController.waitForCoreNodeName look for our core node name
> [junit4:junit4] 2> 375345 T628 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
> [junit4:junit4] 2> 375345 T628 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:35832_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
> [junit4:junit4] 2> 376316 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 376317 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
> [junit4:junit4] 2> 376318 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 376318 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 376355 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376355 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376355 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376355 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376355 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376355 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 376889 T688 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
> [junit4:junit4] 2> 376889 T688 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1
> [junit4:junit4] 2> 376889 T688 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 376890 T688 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 376891 T688 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 376893 T688 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/'
> [junit4:junit4] 2> 376894 T688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/lib/README' to classloader
> [junit4:junit4] 2> 376895 T688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 376960 T688 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 377023 T688 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 377126 T688 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
> [junit4:junit4] 2> 377133 T688 oass.IndexSchema.readSchema [collection1] Schema name=test
> [junit4:junit4] 2> ASYNC NEW_CORE C625 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C625_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
> [junit4:junit4] 2> 377347 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24273/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 377347 T673 C625 P35832 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35832 START replicas=[http://127.0.0.1:24273/collection1/] nUpdates=100
> [junit4:junit4] 2> 377348 T673 C625 P35832 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 377348 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 377349 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 377349 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 377349 T673 C625 P35832 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 377350 T673 C625 P35832 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24273/collection1/. core=collection1
> [junit4:junit4] 2> 377350 T673 C625 P35832 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C626 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C626_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
> [junit4:junit4] 2> 377358 T629 C626 P24273 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 377364 T630 C626 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 377367 T630 C626 P24273 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 377367 T630 C626 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 377368 T630 C626 P24273 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 377369 T630 C626 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@4e1221c3 realtime
> [junit4:junit4] 2> 377369 T630 C626 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 377370 T630 C626 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 377371 T673 C625 P35832 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 377371 T673 C625 P35832 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 377373 T631 C626 P24273 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 377374 T631 C626 P24273 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
> [junit4:junit4] 2> 377375 T673 C625 P35832 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 377375 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 377375 T673 C625 P35832 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 377375 T673 C625 P35832 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 377377 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 377861 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 377862 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node3",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
> [junit4:junit4] 2> 377877 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377878 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377878 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377878 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377878 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377878 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 377938 T688 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 377954 T688 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 377958 T688 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 377978 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 377984 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 377990 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 377991 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 377992 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 377992 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 377994 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 377994 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
> [junit4:junit4] 2> 377994 T688 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 377995 T688 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/
> [junit4:junit4] 2> 377995 T688 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
> [junit4:junit4] 2> 377996 T688 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4
> [junit4:junit4] 2> 377997 T688 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index/
> [junit4:junit4] 2> 377997 T688 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 377998 T688 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index
> [junit4:junit4] 2> 378002 T688 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f823651 lockFactory=org.apache.lucene.store.NativeFSLockFactory@617d6080),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 378002 T688 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 378006 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 378006 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 378007 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 378008 T688 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 378009 T688 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 378009 T688 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 378010 T688 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 378011 T688 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 378011 T688 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 378029 T688 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 378038 T688 oass.SolrIndexSearcher.<init> Opening Searcher@4d9408e4 main
> [junit4:junit4] 2> 378040 T688 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 378040 T688 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 378045 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d9408e4 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 378048 T688 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 378049 T688 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:20834 collection:collection1 shard:shard2
> [junit4:junit4] 2> 378053 T688 oasc.ZkController.register We are http://127.0.0.1:20834/collection1/ and leader is http://127.0.0.1:17310/collection1/
> [junit4:junit4] 2> 378053 T688 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:20834
> [junit4:junit4] 2> 378053 T688 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 378053 T688 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C627 name=collection1 org.apache.solr.core.SolrCore@1aa5f03e url=http://127.0.0.1:20834/collection1 node=127.0.0.1:20834_ C627_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
> [junit4:junit4] 2> 378054 T691 C627 P20834 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 378055 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 378055 T688 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 378055 T691 C627 P20834 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 378055 T691 C627 P20834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 378056 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 378057 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 378057 T691 C627 P20834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 378057 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 378079 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 378080 T644 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
> [junit4:junit4] 2> 378081 T596 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
> [junit4:junit4] 2> 378082 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 379084 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 379384 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 379385 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
> [junit4:junit4] 2> 379428 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 379428 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 379428 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 379428 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 379428 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 379428 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 380083 T644 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
> [junit4:junit4] 2> 380083 T644 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:20834_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
> [junit4:junit4] 2> 380087 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 381089 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> C627_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
> [junit4:junit4] 2> 382085 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:17310/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 382085 T691 C627 P20834 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:20834 START replicas=[http://127.0.0.1:17310/collection1/] nUpdates=100
> [junit4:junit4] 2> 382086 T691 C627 P20834 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 382086 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 382086 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 382087 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 382087 T691 C627 P20834 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 382088 T691 C627 P20834 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:17310/collection1/. core=collection1
> [junit4:junit4] 2> 382088 T691 C627 P20834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 382092 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> ASYNC NEW_CORE C628 name=collection1 org.apache.solr.core.SolrCore@402c34a7 url=http://127.0.0.1:17310/collection1 node=127.0.0.1:17310_ C628_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:17310_, base_url=http://127.0.0.1:17310, leader=true}
> [junit4:junit4] 2> 382095 T645 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
> [junit4:junit4] 2> 382101 T646 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 382104 T646 C628 P17310 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@555673be lockFactory=org.apache.lucene.store.NativeFSLockFactory@29590fb3),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 382105 T646 C628 P17310 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 382105 T646 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 382106 T646 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@750e809d realtime
> [junit4:junit4] 2> 382106 T646 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 382107 T646 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 382108 T691 C627 P20834 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 382108 T691 C627 P20834 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 382110 T647 C628 P17310 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 382110 T647 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
> [junit4:junit4] 2> 382111 T691 C627 P20834 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 382111 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 382112 T691 C627 P20834 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 382112 T691 C627 P20834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 382114 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 382436 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 382437 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"core_node4",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "shard_state":"active",
> [junit4:junit4] 2> "core":"collection1",
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard_range":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
> [junit4:junit4] 2> 382450 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 382450 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 382450 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 382450 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 382450 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 382450 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4] 2> 383095 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383096 T596 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
> [junit4:junit4] 2> ASYNC NEW_CORE C629 name=collection1 org.apache.solr.core.SolrCore@1be13570 url=http://127.0.0.1:14091/collection1 node=127.0.0.1:14091_ C629_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:14091_, base_url=http://127.0.0.1:14091, leader=true}
> [junit4:junit4] 2> 383109 T608 C629 P14091 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 383110 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 383112 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@6761a9a4 realtime
> [junit4:junit4] 2> 383113 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439091209926606848)} 0 8
> [junit4:junit4] 2> ASYNC NEW_CORE C630 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C630_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
> [junit4:junit4] 2> 383141 T660 C630 P35832 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 383142 T660 C630 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 383144 T660 C630 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@52cf3551 realtime
> [junit4:junit4] 2> C627_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
> [junit4:junit4] 2> 383144 T677 C627 P20834 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f823651 lockFactory=org.apache.lucene.store.NativeFSLockFactory@617d6080),segFN=segments_1,generation=1}
> [junit4:junit4] 2> 383145 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439091209947578368&update.from=http://127.0.0.1:24273/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209947578368)} 0 6
> [junit4:junit4] 2> 383145 T677 C627 P20834 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
> [junit4:junit4] 2> 383148 T677 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@7ebe9052 realtime
> [junit4:junit4] 2> 383148 T677 C627 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439091209951772672&update.from=http://127.0.0.1:17310/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209951772672)} 0 6
> [junit4:junit4] 2> 383149 T648 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209951772672)} 0 19
> [junit4:junit4] 2> ASYNC NEW_CORE C631 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C631_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
> [junit4:junit4] 2> 383150 T632 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439091209947578368)} 0 24
> [junit4:junit4] 2> 383152 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383154 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383157 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 383159 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383161 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383164 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383166 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383171 T608 C629 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383176 T608 C629 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 383177 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 383212 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@396ae863 main
> [junit4:junit4] 2> 383213 T608 C629 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383214 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@396ae863 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
> [junit4:junit4] 2> 383215 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091209992667136)],commit=} 0 46
> [junit4:junit4] 2> 383224 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210045095936)]} 0 1
> [junit4:junit4] 2> 383225 T633 C631 P24273 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 383227 T634 C631 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383228 T679 C627 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383229 T648 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383228 T660 C630 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383229 T648 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383229 T679 C627 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383230 T648 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@3ccf0a31 main
> [junit4:junit4] 2> 383231 T679 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@319bf18c main
> [junit4:junit4] 2> 383231 T648 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383232 T679 C627 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383233 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ccf0a31 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383233 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@319bf18c main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383234 T634 C631 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 383234 T679 C627 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> 383234 T648 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 383235 T660 C630 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_2,generation=2}
> [junit4:junit4] 2> 383234 T634 C631 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 383236 T660 C630 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
> [junit4:junit4] 2> 383239 T634 C631 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@3978e480 main
> [junit4:junit4] 2> 383239 T660 C630 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@368bfc86 main
> [junit4:junit4] 2> 383240 T634 C631 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383240 T660 C630 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383241 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3978e480 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
> [junit4:junit4] 2> 383241 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@368bfc86 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
> [junit4:junit4] 2> 383242 T634 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
> [junit4:junit4] 2> 383242 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
> [junit4:junit4] 2> 383243 T633 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210045095936)],commit=} 0 24
> [junit4:junit4] 2> 383245 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383246 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383263 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=13
> [junit4:junit4] 2> 383267 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383274 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383278 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383282 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 383283 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383285 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383293 T628 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383293 T650 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383298 T628 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
> [junit4:junit4] 2> 383299 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=10
> [junit4:junit4] 2> 383305 T630 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383305 T644 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383308 T630 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383310 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383311 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383318 T646 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383318 T705 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383322 T705 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383323 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383330 T708 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383330 T709 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383334 T708 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383335 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 383341 T608 C629 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383347 T608 C629 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 383347 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 383350 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@9556b8 main
> [junit4:junit4] 2> 383351 T608 C629 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383352 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9556b8 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
> [junit4:junit4] 2> 383353 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210170925056)],commit=} 0 14
> [junit4:junit4] 2> 383404 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210189799424)]} 0 42
> [junit4:junit4] 2> 383405 T710 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:17310/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210189799424)]} 0 48
> [junit4:junit4] 2> 383406 T649 C628 P17310 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 383410 T710 C631 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383410 T677 C627 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383410 T714 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383410 T662 C630 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383412 T714 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383411 T677 C627 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383414 T714 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@7d360067 main
> [junit4:junit4] 2> 383414 T677 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@82efb main
> [junit4:junit4] 2> 383415 T714 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383415 T677 C627 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383416 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d360067 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383416 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@82efb main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383417 T714 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> ASYNC NEW_CORE C632 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C632_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
> [junit4:junit4] 2> 383417 T710 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_3,generation=3}
> [junit4:junit4] 2> ASYNC NEW_CORE C633 name=collection1 org.apache.solr.core.SolrCore@1aa5f03e url=http://127.0.0.1:20834/collection1 node=127.0.0.1:20834_ C633_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
> [junit4:junit4] 2> 383417 T677 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
> [junit4:junit4] 2> ASYNC NEW_CORE C634 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C634_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
> [junit4:junit4] 2> 383437 T662 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_2,generation=2}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_3,generation=3}
> [junit4:junit4] 2> 383436 T710 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 383438 T662 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
> [junit4:junit4] 2> 383440 T710 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@46eb25a4 main
> [junit4:junit4] 2> 383441 T662 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@3e4cafcf main
> [junit4:junit4] 2> 383441 T710 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383442 T662 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383443 T672 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
> [junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
> [junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
> [junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
> [junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
> [junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
> [junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
> [junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
> [junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
> [junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:309)
> [junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
> [junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
> [junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
> [junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] 2>
> [junit4:junit4] 2> 383443 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46eb25a4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
> [junit4:junit4] 2> 383444 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e4cafcf main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
> [junit4:junit4] 2> 383445 T710 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
> [junit4:junit4] 2> 383445 T662 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
> [junit4:junit4] 2> ASYNC NEW_CORE C635 name=collection1 org.apache.solr.core.SolrCore@402c34a7 url=http://127.0.0.1:17310/collection1 node=127.0.0.1:17310_ C635_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:17310_, base_url=http://127.0.0.1:17310, leader=true}
> [junit4:junit4] 2> 383446 T649 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 91
> [junit4:junit4] 2> 383447 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383449 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383454 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383458 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383460 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383463 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383466 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383468 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383470 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383476 T663 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383478 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383482 T663 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
> [junit4:junit4] 2> 383483 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 383488 T664 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383489 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383492 T664 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383493 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 383495 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383500 T665 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383502 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 383505 T665 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383506 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383512 T666 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383512 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383516 T666 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383517 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> ASYNC NEW_CORE C636 name=collection1 org.apache.solr.core.SolrCore@1be13570 url=http://127.0.0.1:14091/collection1 node=127.0.0.1:14091_ C636_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:14091_, base_url=http://127.0.0.1:14091, leader=true}
> [junit4:junit4] 2> 383523 T608 C636 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383529 T608 C636 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 383530 T608 C636 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 383532 T608 C636 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@73dabf20 main
> [junit4:junit4] 2> 383533 T608 C636 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383534 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73dabf20 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
> [junit4:junit4] 2> 383535 T608 C636 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210361765888)],commit=} 0 14
> [junit4:junit4] 2> 383547 T660 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210382737408)]} 0 2
> [junit4:junit4] 2> 383548 T716 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35832/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210382737408)]} 0 8
> [junit4:junit4] 2> 383549 T661 C634 P35832 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 383552 T716 C632 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383553 T680 C633 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383553 T721 C635 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383552 T720 C634 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383554 T721 C635 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383553 T680 C633 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383555 T721 C635 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@c505069 main
> [junit4:junit4] 2> 383556 T680 C633 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@533045a7 main
> [junit4:junit4] 2> 383556 T721 C635 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383557 T680 C633 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383557 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c505069 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383557 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@533045a7 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383558 T721 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 383558 T680 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 383558 T716 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 383560 T720 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_3,generation=3}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_4,generation=4}
> [junit4:junit4] 2> 383560 T716 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 383560 T720 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
> [junit4:junit4] 2> 383563 T716 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@4b978a7b main
> [junit4:junit4] 2> 383563 T720 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@45325271 main
> [junit4:junit4] 2> 383564 T716 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383565 T720 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383565 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b978a7b main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
> [junit4:junit4] 2> 383565 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45325271 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
> [junit4:junit4] 2> 383566 T716 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
> [junit4:junit4] 2> 383566 T720 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
> [junit4:junit4] 2> 383567 T661 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
> [junit4:junit4] 2> 383569 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383571 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383575 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383579 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383580 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383584 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383587 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383589 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383590 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383597 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383597 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383601 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383603 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 383609 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383609 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383614 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383615 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 383616 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383623 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383624 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383628 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383629 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
> [junit4:junit4] 2> 383637 T708 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383637 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
> [junit4:junit4] 2> 383640 T708 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383641 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383647 T608 C636 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383653 T608 C636 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 383654 T608 C636 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 383656 T608 C636 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@578f989f main
> [junit4:junit4] 2> 383657 T608 C636 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383658 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@578f989f main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
> [junit4:junit4] 2> 383659 T608 C636 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210491789312)],commit=} 0 14
> [junit4:junit4] 2> 383671 T660 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210512760832)]} 0 2
> [junit4:junit4] 2> 383672 T723 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20834/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210512760832)]} 0 7
> [junit4:junit4] 2> 383673 T678 C633 P20834 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 383677 T723 C632 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383677 T728 C635 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383677 T727 C634 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383677 T681 C633 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 383678 T728 C635 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383679 T681 C633 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
> [junit4:junit4] 2> 383680 T728 C635 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@5608e1d9 main
> [junit4:junit4] 2> 383680 T681 C633 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@80e140a main
> [junit4:junit4] 2> 383681 T728 C635 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383681 T681 C633 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383682 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5608e1d9 main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383682 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@80e140a main{StandardDirectoryReader(segments_1:1:nrt)}
> [junit4:junit4] 2> 383683 T728 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 383683 T723 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 383683 T681 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
> [junit4:junit4] 2> 383684 T723 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 383684 T727 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_4,generation=4}
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_5,generation=5}
> [junit4:junit4] 2> 383685 T727 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
> [junit4:junit4] 2> 383687 T723 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@31bff073 main
> [junit4:junit4] 2> 383688 T723 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383688 T727 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@7886d870 main
> [junit4:junit4] 2> 383689 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31bff073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
> [junit4:junit4] 2> 383690 T727 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 383690 T723 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
> [junit4:junit4] 2> 383690 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7886d870 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
> [junit4:junit4] 2> 383691 T727 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
> [junit4:junit4] 2> 383692 T678 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 30
> [junit4:junit4] 2> 383694 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383696 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383701 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383705 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
> [junit4:junit4] 2> 383706 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383710 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383713 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383715 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383717 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383723 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383723 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383727 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383728 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383736 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383736 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383741 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383742 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=10
> [junit4:junit4] 2> 383744 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 383750 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 383750 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 383754 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
> [junit4:junit4] 2> 383755 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
> [junit4:junit4] 2> 383761 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326525&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true
>
> [...truncated too long message...]
>
> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {foo_b=Lucene41(blocksize=128), n_f1=PostingsFormat(name=Direct), cat=PostingsFormat(name=MockSep), foo_d=MockVariableIntBlock(baseBlockSize=81), n_tl1=PostingsFormat(name=Direct), foo_f=Lucene41(blocksize=128), n_d1=PostingsFormat(name=MockSep), rnd_b=MockVariableIntBlock(baseBlockSize=81), intDefault=PostingsFormat(name=MockSep), n_td1=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=MockSep), id=MockVariableIntBlock(baseBlockSize=81), range_facet_sl=Lucene41(blocksize=128), range_facet_si=PostingsFormat(name=Direct), oddField_s=Lucene41(blocksize=128), sequence_i=Lucene41(blocksize=128), name=MockVariableIntBlock(baseBlockSize=81), foo_i=PostingsFormat(name=Direct), regex_dup_B_s=PostingsFormat(name=MockSep), multiDefault=PostingsFormat(name=Direct), n_tf1=PostingsFormat(name=MockSep), n_dt1=PostingsFormat(name=Direct), genre_s=Lucene41(blocksize=128), author_t=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=MockSep), n_ti1=MockVariableIntBlock(baseBlockSize=81), text=Lucene41(blocksize=128), _version_=PostingsFormat(name=MockSep), val_i=MockVariableIntBlock(baseBlockSize=81), SubjectTerms_mfacet=PostingsFormat(name=MockSep), series_t=PostingsFormat(name=Direct), a_t=PostingsFormat(name=MockSep), n_tdt1=PostingsFormat(name=MockSep), regex_dup_A_s=Lucene41(blocksize=128), price=Lucene41(blocksize=128), other_tl1=PostingsFormat(name=MockSep), n_l1=PostingsFormat(name=MockSep), a_si=PostingsFormat(name=Direct), inStock=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=es_US, timezone=America/Guyana
> [junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=96278400,total=336592896
> [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestDocSet, CSVRequestHandlerTest, TestShardHandlerFactory, TestLRUCache, TestRandomDVFaceting, ReturnFieldsTest, StatelessScriptUpdateProcessorFactoryTest, SliceStateTest, TestSweetSpotSimilarityFactory, PrimUtilsTest, ChaosMonkeyNothingIsSafeTest, TestQuerySenderListener, TestManagedSchemaFieldResource, OverseerCollectionProcessorTest, ResponseLogComponentTest, TestPluginEnable, TestCoreContainer, ClusterStateUpdateTest, ParsingFieldUpdateProcessorsTest, BasicDistributedZkTest]
> [junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=C78A344FD79A7AD2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=America/Guyana -Dtests.file.encoding=ISO-8859-1
> [junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
> [junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
> [junit4:junit4] > 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
> [junit4:junit4] > 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> [junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> [junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
> [junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
> [junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
> [junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> [junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4] > at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
> [junit4:junit4] Completed on J0 in 217.75s, 1 test, 2 errors <<< FAILURES!
>
> [...truncated 779 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 849 ignored (5 assumptions)
>
> Total time: 38 minutes 30 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4095 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4095/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest: 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391) at java.net.Socket.connect(Socket.java:579) at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805) at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180) at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:722)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
at java.net.Socket.connect(Socket.java:579)
at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
Build Log:
[...truncated 9329 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4] 2> 351088 T596 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 351096 T596 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1372424293864
[junit4:junit4] 2> 351097 T596 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 351098 T597 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 351199 T596 oasc.ZkTestServer.run start zk server on port:36823
[junit4:junit4] 2> 351200 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 351206 T603 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@215dda5b name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 351207 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 351207 T596 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 351212 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 351221 T605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@287cbb41 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 351221 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 351222 T596 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 351225 T596 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 351234 T596 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 351237 T596 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 351240 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 351241 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 351247 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 351248 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 351353 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 351354 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 351358 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 351359 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 351368 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 351369 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 351373 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 351374 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 351378 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 351378 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 351382 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 351383 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 351386 T596 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 351387 T596 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 351662 T596 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 351666 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14091
[junit4:junit4] 2> 351667 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 351667 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 351668 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158
[junit4:junit4] 2> 351669 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/solr.xml
[junit4:junit4] 2> 351669 T596 oasc.CoreContainer.<init> New CoreContainer 52426110
[junit4:junit4] 2> 351670 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/'
[junit4:junit4] 2> 351670 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/'
[junit4:junit4] 2> 351817 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 351818 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 351819 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 351819 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 351820 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 351820 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 351821 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 351821 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 351822 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 351822 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 351842 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 351843 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
[junit4:junit4] 2> 351843 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 351844 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 351848 T616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bc90bce name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 351848 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 351858 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 351872 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 351874 T618 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43ec9e1e name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 351875 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 351882 T596 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 351888 T596 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 351893 T596 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 351895 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14091_
[junit4:junit4] 2> 351902 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14091_
[junit4:junit4] 2> 351907 T596 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 351924 T596 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 351928 T596 oasc.Overseer.start Overseer (id=89943198522998787-127.0.0.1:14091_-n_0000000000) starting
[junit4:junit4] 2> 351940 T596 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 351953 T620 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 351954 T596 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 351957 T596 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 351960 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 351964 T619 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 351968 T621 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 351968 T621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 351970 T621 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 353469 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 353470 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:14091_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:14091"}
[junit4:junit4] 2> 353471 T619 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 353471 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 353486 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 353972 T621 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 353972 T621 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1
[junit4:junit4] 2> 353972 T621 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 353973 T621 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 353973 T621 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 353975 T621 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/'
[junit4:junit4] 2> 353977 T621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 353978 T621 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/lib/README' to classloader
[junit4:junit4] 2> 354031 T621 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 354093 T621 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 354196 T621 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 354203 T621 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 354863 T621 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 354876 T621 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 354880 T621 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 354900 T621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 354905 T621 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 354910 T621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 354911 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 354912 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 354912 T621 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 354913 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 354914 T621 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 354914 T621 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 354915 T621 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1372424294158/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/
[junit4:junit4] 2> 354915 T621 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
[junit4:junit4] 2> 354916 T621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data
[junit4:junit4] 2> 354916 T621 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index/
[junit4:junit4] 2> 354917 T621 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 354917 T621 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/control/data/index
[junit4:junit4] 2> 354922 T621 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 354922 T621 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 354925 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 354925 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 354926 T621 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 354927 T621 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 354928 T621 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 354928 T621 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 354929 T621 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 354944 T621 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 354951 T621 oass.SolrIndexSearcher.<init> Opening Searcher@14639129 main
[junit4:junit4] 2> 354952 T621 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 354953 T621 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 354957 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14639129 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 354959 T621 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 354959 T621 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14091 collection:control_collection shard:shard1
[junit4:junit4] 2> 354960 T621 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 354981 T621 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 354993 T621 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 354993 T621 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:14091/collection1/
[junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 354994 T621 oasc.SyncStrategy.syncToMe http://127.0.0.1:14091/collection1/ has no replicas
[junit4:junit4] 2> 354995 T621 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:14091/collection1/ shard1
[junit4:junit4] 2> 354995 T621 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 356496 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 356525 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 356542 T621 oasc.ZkController.register We are http://127.0.0.1:14091/collection1/ and leader is http://127.0.0.1:14091/collection1/
[junit4:junit4] 2> 356542 T621 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14091
[junit4:junit4] 2> 356542 T621 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 356543 T621 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 356543 T621 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 356545 T621 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 356546 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 356547 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 356548 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 356567 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 356569 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 356571 T625 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f14acc4 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 356572 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 356574 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 356577 T596 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 356848 T596 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 356851 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:24273
[junit4:junit4] 2> 356851 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 356852 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 356853 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344
[junit4:junit4] 2> 356853 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/solr.xml
[junit4:junit4] 2> 356854 T596 oasc.CoreContainer.<init> New CoreContainer 1036840427
[junit4:junit4] 2> 356854 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/'
[junit4:junit4] 2> 356855 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/'
[junit4:junit4] 2> 356989 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 356990 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 356991 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 356991 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 356992 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 356993 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 356993 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 356994 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 356994 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 356995 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 357012 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 357013 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
[junit4:junit4] 2> 357014 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 357015 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 357019 T636 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3eb79d53 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 357019 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 357033 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 357045 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 357048 T638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d692086 name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 357048 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 357060 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 358041 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 358042 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:14091_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:14091"}
[junit4:junit4] 2> 358059 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358059 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358059 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358064 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:24273_
[junit4:junit4] 2> 358066 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:24273_
[junit4:junit4] 2> 358073 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358074 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 358074 T638 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 358074 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 358076 T625 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 358076 T618 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 358083 T639 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 358083 T639 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 358085 T639 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 359566 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 359567 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24273_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24273"}
[junit4:junit4] 2> 359568 T619 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 359568 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 359582 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359582 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 359582 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 360087 T639 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 360087 T639 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1
[junit4:junit4] 2> 360087 T639 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 360088 T639 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 360088 T639 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 360090 T639 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/'
[junit4:junit4] 2> 360092 T639 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/lib/README' to classloader
[junit4:junit4] 2> 360093 T639 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 360158 T639 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 360220 T639 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 360323 T639 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 360332 T639 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 361000 T639 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 361014 T639 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 361018 T639 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 361039 T639 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 361044 T639 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 361049 T639 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 361051 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 361051 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 361052 T639 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 361053 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 361053 T639 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 361054 T639 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 361054 T639 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1372424299344/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/
[junit4:junit4] 2> 361055 T639 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
[junit4:junit4] 2> 361056 T639 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1
[junit4:junit4] 2> 361056 T639 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index/
[junit4:junit4] 2> 361056 T639 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 361057 T639 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty1/index
[junit4:junit4] 2> 361061 T639 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> 361061 T639 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 361065 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 361065 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 361066 T639 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 361067 T639 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 361068 T639 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 361069 T639 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 361070 T639 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 361086 T639 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 361094 T639 oass.SolrIndexSearcher.<init> Opening Searcher@274be5a4 main
[junit4:junit4] 2> 361095 T639 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 361095 T639 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 361101 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@274be5a4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 361104 T639 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 361104 T639 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:24273 collection:collection1 shard:shard1
[junit4:junit4] 2> 361105 T639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 361132 T639 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 361137 T639 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 361137 T639 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 361137 T639 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:24273/collection1/
[junit4:junit4] 2> 361138 T639 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 361138 T639 oasc.SyncStrategy.syncToMe http://127.0.0.1:24273/collection1/ has no replicas
[junit4:junit4] 2> 361138 T639 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:24273/collection1/ shard1
[junit4:junit4] 2> 361139 T639 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 362590 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 362621 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362621 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362621 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 362635 T639 oasc.ZkController.register We are http://127.0.0.1:24273/collection1/ and leader is http://127.0.0.1:24273/collection1/
[junit4:junit4] 2> 362635 T639 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:24273
[junit4:junit4] 2> 362635 T639 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 362636 T639 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 362636 T639 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 362638 T639 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 362639 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 362640 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 362641 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 362975 T596 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 362979 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:17310
[junit4:junit4] 2> 362981 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 362981 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 362982 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430
[junit4:junit4] 2> 362982 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/solr.xml
[junit4:junit4] 2> 362983 T596 oasc.CoreContainer.<init> New CoreContainer 14294857
[junit4:junit4] 2> 362984 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/'
[junit4:junit4] 2> 362984 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/'
[junit4:junit4] 2> 363136 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 363137 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 363138 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 363138 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 363139 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 363139 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 363140 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 363140 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 363141 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 363142 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 363160 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 363161 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
[junit4:junit4] 2> 363162 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 363163 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 363166 T652 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51e23255 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 363167 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 363177 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 363190 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 363192 T654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6fce68d name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 363193 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 363203 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 364129 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 364130 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:24273_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:24273"}
[junit4:junit4] 2> 364148 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364148 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364148 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364148 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364207 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:17310_
[junit4:junit4] 2> 364210 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:17310_
[junit4:junit4] 2> 364212 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364213 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364212 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 364213 T638 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 364214 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 364215 T625 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 364215 T618 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 364216 T654 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 364222 T655 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 364222 T655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 364224 T655 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 365656 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 365657 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:17310_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:17310"}
[junit4:junit4] 2> 365658 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 365658 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 365685 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365685 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365685 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 365685 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 366226 T655 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 366226 T655 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1
[junit4:junit4] 2> 366226 T655 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 366227 T655 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 366227 T655 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 366229 T655 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/'
[junit4:junit4] 2> 366231 T655 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/lib/README' to classloader
[junit4:junit4] 2> 366232 T655 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 366298 T655 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 366362 T655 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 366464 T655 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 366472 T655 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 367149 T655 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 367165 T655 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 367169 T655 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 367190 T655 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 367196 T655 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 367201 T655 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 367203 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 367203 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 367204 T655 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 367205 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 367205 T655 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 367206 T655 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 367206 T655 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1372424305430/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/
[junit4:junit4] 2> 367207 T655 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
[junit4:junit4] 2> 367208 T655 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2
[junit4:junit4] 2> 367208 T655 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index/
[junit4:junit4] 2> 367208 T655 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 367209 T655 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty2/index
[junit4:junit4] 2> 367213 T655 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@555673be lockFactory=org.apache.lucene.store.NativeFSLockFactory@29590fb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 367214 T655 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 367217 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 367218 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 367218 T655 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 367219 T655 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 367220 T655 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 367220 T655 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 367221 T655 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 367222 T655 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 367222 T655 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 367242 T655 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 367251 T655 oass.SolrIndexSearcher.<init> Opening Searcher@50776c5f main
[junit4:junit4] 2> 367252 T655 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 367252 T655 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 367258 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@50776c5f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 367261 T655 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 367261 T655 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:17310 collection:collection1 shard:shard2
[junit4:junit4] 2> 367262 T655 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 367302 T655 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 367313 T655 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 367313 T655 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:17310/collection1/
[junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 367314 T655 oasc.SyncStrategy.syncToMe http://127.0.0.1:17310/collection1/ has no replicas
[junit4:junit4] 2> 367315 T655 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:17310/collection1/ shard2
[junit4:junit4] 2> 367315 T655 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 368693 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 368712 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 368712 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 368712 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 368712 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 368759 T655 oasc.ZkController.register We are http://127.0.0.1:17310/collection1/ and leader is http://127.0.0.1:17310/collection1/
[junit4:junit4] 2> 368759 T655 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:17310
[junit4:junit4] 2> 368759 T655 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 368759 T655 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 368759 T655 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 368761 T655 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 368763 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 368763 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 368764 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 369048 T596 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 369052 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35832
[junit4:junit4] 2> 369053 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 369053 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 369054 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550
[junit4:junit4] 2> 369054 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/solr.xml
[junit4:junit4] 2> 369055 T596 oasc.CoreContainer.<init> New CoreContainer 2003196108
[junit4:junit4] 2> 369056 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/'
[junit4:junit4] 2> 369056 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/'
[junit4:junit4] 2> 369188 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 369189 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 369190 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 369190 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 369191 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 369191 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 369192 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 369192 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 369193 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 369193 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 369212 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 369213 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
[junit4:junit4] 2> 369214 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 369215 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 369218 T668 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d6e8b68 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 369219 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 369236 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 369247 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 369250 T670 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4aca485d name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 369250 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 369261 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 370226 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 370227 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:17310_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:17310"}
[junit4:junit4] 2> 370241 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370241 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370241 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370241 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370241 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370265 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35832_
[junit4:junit4] 2> 370267 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35832_
[junit4:junit4] 2> 370270 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370270 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370270 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 370271 T670 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 370271 T638 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 370272 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 370272 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 370273 T625 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 370273 T618 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 370274 T654 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 370280 T671 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 370280 T671 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 370282 T671 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 371748 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 371749 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
[junit4:junit4] 2> 371750 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 371750 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 371780 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 371781 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 371781 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 371781 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 371780 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 372284 T671 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 372284 T671 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1
[junit4:junit4] 2> 372284 T671 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 372285 T671 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 372286 T671 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 372288 T671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/'
[junit4:junit4] 2> 372289 T671 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/lib/README' to classloader
[junit4:junit4] 2> 372290 T671 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 372355 T671 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 372418 T671 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 372520 T671 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 372527 T671 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 373195 T671 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 373210 T671 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 373214 T671 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 373234 T671 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 373240 T671 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 373245 T671 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 373246 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 373247 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 373247 T671 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 373249 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 373249 T671 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 373249 T671 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 373250 T671 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1372424311550/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/
[junit4:junit4] 2> 373250 T671 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
[junit4:junit4] 2> 373251 T671 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3
[junit4:junit4] 2> 373252 T671 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index/
[junit4:junit4] 2> 373252 T671 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 373252 T671 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty3/index
[junit4:junit4] 2> 373256 T671 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
[junit4:junit4] 2> 373257 T671 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 373260 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 373261 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 373262 T671 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 373263 T671 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 373263 T671 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 373264 T671 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 373264 T671 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 373265 T671 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 373265 T671 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 373283 T671 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 373292 T671 oass.SolrIndexSearcher.<init> Opening Searcher@40962cea main
[junit4:junit4] 2> 373293 T671 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 373294 T671 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 373299 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40962cea main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 373302 T671 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 373303 T671 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35832 collection:collection1 shard:shard1
[junit4:junit4] 2> 373313 T671 oasc.ZkController.register We are http://127.0.0.1:35832/collection1/ and leader is http://127.0.0.1:24273/collection1/
[junit4:junit4] 2> 373313 T671 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35832
[junit4:junit4] 2> 373314 T671 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 373314 T671 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C624 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C624_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
[junit4:junit4] 2> 373314 T673 C624 P35832 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 373315 T673 C624 P35832 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 373315 T671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 373315 T673 C624 P35832 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 373316 T673 C624 P35832 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 373317 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 373317 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 373318 T673 C624 P35832 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 373318 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 373342 T628 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 373643 T596 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 373646 T596 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:20834
[junit4:junit4] 2> 373647 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 373648 T596 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 373648 T596 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108
[junit4:junit4] 2> 373649 T596 oasc.CoreContainer$Initializer.initialize looking for solr config file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/solr.xml
[junit4:junit4] 2> 373649 T596 oasc.CoreContainer.<init> New CoreContainer 1561369995
[junit4:junit4] 2> 373650 T596 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/'
[junit4:junit4] 2> 373651 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/'
[junit4:junit4] 2> 373798 T596 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 373799 T596 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 373799 T596 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 373800 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 373800 T596 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 373801 T596 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 373801 T596 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 373802 T596 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 373802 T596 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 373803 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 373822 T596 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 373822 T596 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36823/solr
[junit4:junit4] 2> 373823 T596 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 373824 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 373827 T685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f7cd760 name:ZooKeeperConnection Watcher:127.0.0.1:36823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 373828 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 373844 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 373857 T596 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 373859 T687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3df988fd name:ZooKeeperConnection Watcher:127.0.0.1:36823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 373860 T596 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 373866 T596 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 374795 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 374796 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
[junit4:junit4] 2> 374809 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374810 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374810 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374810 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374810 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374810 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374870 T596 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:20834_
[junit4:junit4] 2> 374872 T596 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:20834_
[junit4:junit4] 2> 374875 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374875 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374875 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374876 T670 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374875 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 374876 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 374876 T638 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374877 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 374877 T687 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374878 T618 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374878 T654 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374879 T625 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 374885 T688 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 374885 T688 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 374887 T688 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 375345 T628 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 375345 T628 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:35832_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 376316 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 376317 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
[junit4:junit4] 2> 376318 T619 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 376318 T619 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 376355 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376355 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376355 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376355 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376355 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376355 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 376889 T688 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 376889 T688 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1
[junit4:junit4] 2> 376889 T688 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 376890 T688 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 376891 T688 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 376893 T688 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/'
[junit4:junit4] 2> 376894 T688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/lib/README' to classloader
[junit4:junit4] 2> 376895 T688 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 376960 T688 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 377023 T688 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 377126 T688 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 377133 T688 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C625 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C625_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
[junit4:junit4] 2> 377347 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:24273/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 377347 T673 C625 P35832 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:35832 START replicas=[http://127.0.0.1:24273/collection1/] nUpdates=100
[junit4:junit4] 2> 377348 T673 C625 P35832 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 377348 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 377349 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 377349 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 377349 T673 C625 P35832 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 377350 T673 C625 P35832 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:24273/collection1/. core=collection1
[junit4:junit4] 2> 377350 T673 C625 P35832 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C626 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C626_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
[junit4:junit4] 2> 377358 T629 C626 P24273 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 377364 T630 C626 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 377367 T630 C626 P24273 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> 377367 T630 C626 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 377368 T630 C626 P24273 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 377369 T630 C626 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@4e1221c3 realtime
[junit4:junit4] 2> 377369 T630 C626 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 377370 T630 C626 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 377371 T673 C625 P35832 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 377371 T673 C625 P35832 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 377373 T631 C626 P24273 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 377374 T631 C626 P24273 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 377375 T673 C625 P35832 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 377375 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 377375 T673 C625 P35832 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 377375 T673 C625 P35832 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 377377 T673 C625 P35832 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 377861 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 377862 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:35832_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:35832"}
[junit4:junit4] 2> 377877 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377878 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377878 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377878 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377878 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377878 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 377938 T688 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 377954 T688 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 377958 T688 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 377978 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 377984 T688 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 377990 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 377991 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 377992 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 377992 T688 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 377994 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 377994 T688 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 377994 T688 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 377995 T688 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1372424316108/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/
[junit4:junit4] 2> 377995 T688 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@253be03d
[junit4:junit4] 2> 377996 T688 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4
[junit4:junit4] 2> 377997 T688 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index/
[junit4:junit4] 2> 377997 T688 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 377998 T688 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1372424293863/jetty4/index
[junit4:junit4] 2> 378002 T688 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f823651 lockFactory=org.apache.lucene.store.NativeFSLockFactory@617d6080),segFN=segments_1,generation=1}
[junit4:junit4] 2> 378002 T688 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 378006 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 378006 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 378007 T688 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 378008 T688 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 378009 T688 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 378009 T688 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 378010 T688 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 378011 T688 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 378011 T688 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 378029 T688 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 378038 T688 oass.SolrIndexSearcher.<init> Opening Searcher@4d9408e4 main
[junit4:junit4] 2> 378040 T688 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 378040 T688 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 378045 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d9408e4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 378048 T688 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 378049 T688 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:20834 collection:collection1 shard:shard2
[junit4:junit4] 2> 378053 T688 oasc.ZkController.register We are http://127.0.0.1:20834/collection1/ and leader is http://127.0.0.1:17310/collection1/
[junit4:junit4] 2> 378053 T688 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:20834
[junit4:junit4] 2> 378053 T688 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 378053 T688 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C627 name=collection1 org.apache.solr.core.SolrCore@1aa5f03e url=http://127.0.0.1:20834/collection1 node=127.0.0.1:20834_ C627_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
[junit4:junit4] 2> 378054 T691 C627 P20834 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 378055 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 378055 T688 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 378055 T691 C627 P20834 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 378055 T691 C627 P20834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 378056 T596 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4:junit4] 2> 378057 T596 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 378057 T691 C627 P20834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 378057 T596 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 378079 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 378080 T644 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 378081 T596 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 378082 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 379084 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 379384 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 379385 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
[junit4:junit4] 2> 379428 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379428 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379428 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379428 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379428 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 379428 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 380083 T644 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 380083 T644 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:20834_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2003
[junit4:junit4] 2> 380087 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 381089 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C627_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
[junit4:junit4] 2> 382085 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:17310/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 382085 T691 C627 P20834 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:20834 START replicas=[http://127.0.0.1:17310/collection1/] nUpdates=100
[junit4:junit4] 2> 382086 T691 C627 P20834 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 382086 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 382086 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 382087 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 382087 T691 C627 P20834 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 382088 T691 C627 P20834 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:17310/collection1/. core=collection1
[junit4:junit4] 2> 382088 T691 C627 P20834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 382092 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C628 name=collection1 org.apache.solr.core.SolrCore@402c34a7 url=http://127.0.0.1:17310/collection1 node=127.0.0.1:17310_ C628_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:17310_, base_url=http://127.0.0.1:17310, leader=true}
[junit4:junit4] 2> 382095 T645 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 382101 T646 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 382104 T646 C628 P17310 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@555673be lockFactory=org.apache.lucene.store.NativeFSLockFactory@29590fb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 382105 T646 C628 P17310 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 382105 T646 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 382106 T646 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@750e809d realtime
[junit4:junit4] 2> 382106 T646 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 382107 T646 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 382108 T691 C627 P20834 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 382108 T691 C627 P20834 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 382110 T647 C628 P17310 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 382110 T647 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 382111 T691 C627 P20834 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 382111 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 382112 T691 C627 P20834 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 382112 T691 C627 P20834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 382114 T691 C627 P20834 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 382436 T619 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 382437 T619 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:20834_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:20834"}
[junit4:junit4] 2> 382450 T687 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382450 T654 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382450 T638 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382450 T618 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382450 T625 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 382450 T670 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 383095 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383096 T596 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C629 name=collection1 org.apache.solr.core.SolrCore@1be13570 url=http://127.0.0.1:14091/collection1 node=127.0.0.1:14091_ C629_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:14091_, base_url=http://127.0.0.1:14091, leader=true}
[junit4:junit4] 2> 383109 T608 C629 P14091 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 383110 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 383112 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@6761a9a4 realtime
[junit4:junit4] 2> 383113 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439091209926606848)} 0 8
[junit4:junit4] 2> ASYNC NEW_CORE C630 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C630_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
[junit4:junit4] 2> 383141 T660 C630 P35832 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
[junit4:junit4] 2> 383142 T660 C630 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 383144 T660 C630 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@52cf3551 realtime
[junit4:junit4] 2> C627_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
[junit4:junit4] 2> 383144 T677 C627 P20834 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@f823651 lockFactory=org.apache.lucene.store.NativeFSLockFactory@617d6080),segFN=segments_1,generation=1}
[junit4:junit4] 2> 383145 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439091209947578368&update.from=http://127.0.0.1:24273/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209947578368)} 0 6
[junit4:junit4] 2> 383145 T677 C627 P20834 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 383148 T677 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@7ebe9052 realtime
[junit4:junit4] 2> 383148 T677 C627 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439091209951772672&update.from=http://127.0.0.1:17310/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209951772672)} 0 6
[junit4:junit4] 2> 383149 T648 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439091209951772672)} 0 19
[junit4:junit4] 2> ASYNC NEW_CORE C631 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C631_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
[junit4:junit4] 2> 383150 T632 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439091209947578368)} 0 24
[junit4:junit4] 2> 383152 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383154 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383157 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=1
[junit4:junit4] 2> 383159 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383161 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383164 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383166 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&wt=javabin&version=2&sanity_check=is_empty} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383171 T608 C629 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383176 T608 C629 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_2,generation=2}
[junit4:junit4] 2> 383177 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 383212 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@396ae863 main
[junit4:junit4] 2> 383213 T608 C629 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383214 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@396ae863 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 383215 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091209992667136)],commit=} 0 46
[junit4:junit4] 2> 383224 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210045095936)]} 0 1
[junit4:junit4] 2> 383225 T633 C631 P24273 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 383227 T634 C631 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383228 T679 C627 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383229 T648 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383228 T660 C630 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383229 T648 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383229 T679 C627 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383230 T648 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@3ccf0a31 main
[junit4:junit4] 2> 383231 T679 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@319bf18c main
[junit4:junit4] 2> 383231 T648 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383232 T679 C627 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383233 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ccf0a31 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383233 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@319bf18c main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383234 T634 C631 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_2,generation=2}
[junit4:junit4] 2> 383234 T679 C627 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 383234 T648 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 383235 T660 C630 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_2,generation=2}
[junit4:junit4] 2> 383234 T634 C631 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 383236 T660 C630 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 383239 T634 C631 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@3978e480 main
[junit4:junit4] 2> 383239 T660 C630 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@368bfc86 main
[junit4:junit4] 2> 383240 T634 C631 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383240 T660 C630 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383241 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3978e480 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 383241 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@368bfc86 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):c1)}
[junit4:junit4] 2> 383242 T634 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 383242 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 383243 T633 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210045095936)],commit=} 0 24
[junit4:junit4] 2> 383245 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383246 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383263 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=13
[junit4:junit4] 2> 383267 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383274 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383278 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383282 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 383283 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383285 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383293 T628 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383293 T650 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383298 T628 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326056&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 383299 T633 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=10
[junit4:junit4] 2> 383305 T630 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383305 T644 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383308 T630 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326069&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383310 T661 C630 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383311 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383318 T646 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383318 T705 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383322 T705 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326082&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383323 T649 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383330 T708 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383330 T709 C628 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383334 T708 C631 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326093&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383335 T678 C627 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 383341 T608 C629 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383347 T608 C629 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_3,generation=3}
[junit4:junit4] 2> 383347 T608 C629 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 383350 T608 C629 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@9556b8 main
[junit4:junit4] 2> 383351 T608 C629 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383352 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9556b8 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 383353 T608 C629 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210170925056)],commit=} 0 14
[junit4:junit4] 2> 383404 T660 C630 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210189799424)]} 0 42
[junit4:junit4] 2> 383405 T710 C631 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:17310/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210189799424)]} 0 48
[junit4:junit4] 2> 383406 T649 C628 P17310 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 383410 T710 C631 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383410 T677 C627 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383410 T714 C628 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383410 T662 C630 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383412 T714 C628 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383411 T677 C627 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383414 T714 C628 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@7d360067 main
[junit4:junit4] 2> 383414 T677 C627 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@82efb main
[junit4:junit4] 2> 383415 T714 C628 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383415 T677 C627 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383416 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d360067 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383416 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@82efb main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383417 T714 C628 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> ASYNC NEW_CORE C632 name=collection1 org.apache.solr.core.SolrCore@38398a1a url=http://127.0.0.1:24273/collection1 node=127.0.0.1:24273_ C632_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:24273_, base_url=http://127.0.0.1:24273, leader=true}
[junit4:junit4] 2> 383417 T710 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_3,generation=3}
[junit4:junit4] 2> ASYNC NEW_CORE C633 name=collection1 org.apache.solr.core.SolrCore@1aa5f03e url=http://127.0.0.1:20834/collection1 node=127.0.0.1:20834_ C633_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:20834_, base_url=http://127.0.0.1:20834}
[junit4:junit4] 2> 383417 T677 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> ASYNC NEW_CORE C634 name=collection1 org.apache.solr.core.SolrCore@601c0454 url=http://127.0.0.1:35832/collection1 node=127.0.0.1:35832_ C634_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:35832_, base_url=http://127.0.0.1:35832}
[junit4:junit4] 2> 383437 T662 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_3,generation=3}
[junit4:junit4] 2> 383436 T710 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 383438 T662 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 383440 T710 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@46eb25a4 main
[junit4:junit4] 2> 383441 T662 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@3e4cafcf main
[junit4:junit4] 2> 383441 T710 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383442 T662 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383443 T672 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:type=fieldValueCache,id=org.apache.solr.search.FastLRUCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:309)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 383443 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46eb25a4 main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 383444 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3e4cafcf main{StandardDirectoryReader(segments_3:7:nrt _1(5.0):c1)}
[junit4:junit4] 2> 383445 T710 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4] 2> 383445 T662 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4] 2> ASYNC NEW_CORE C635 name=collection1 org.apache.solr.core.SolrCore@402c34a7 url=http://127.0.0.1:17310/collection1 node=127.0.0.1:17310_ C635_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:17310_, base_url=http://127.0.0.1:17310, leader=true}
[junit4:junit4] 2> 383446 T649 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 91
[junit4:junit4] 2> 383447 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383449 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383454 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383458 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383460 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383463 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383466 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383468 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383470 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383476 T663 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383478 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383482 T663 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326241&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=1
[junit4:junit4] 2> 383483 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 383488 T664 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383489 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383492 T664 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326253&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383493 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=7
[junit4:junit4] 2> 383495 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383500 T665 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383502 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 383505 T665 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326265&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383506 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383512 T666 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383512 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383516 T666 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326276&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383517 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> ASYNC NEW_CORE C636 name=collection1 org.apache.solr.core.SolrCore@1be13570 url=http://127.0.0.1:14091/collection1 node=127.0.0.1:14091_ C636_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:14091_, base_url=http://127.0.0.1:14091, leader=true}
[junit4:junit4] 2> 383523 T608 C636 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383529 T608 C636 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_4,generation=4}
[junit4:junit4] 2> 383530 T608 C636 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 383532 T608 C636 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@73dabf20 main
[junit4:junit4] 2> 383533 T608 C636 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383534 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73dabf20 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 383535 T608 C636 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210361765888)],commit=} 0 14
[junit4:junit4] 2> 383547 T660 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210382737408)]} 0 2
[junit4:junit4] 2> 383548 T716 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:35832/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210382737408)]} 0 8
[junit4:junit4] 2> 383549 T661 C634 P35832 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 383552 T716 C632 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383553 T680 C633 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383553 T721 C635 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383552 T720 C634 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383554 T721 C635 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383553 T680 C633 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383555 T721 C635 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@c505069 main
[junit4:junit4] 2> 383556 T680 C633 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@533045a7 main
[junit4:junit4] 2> 383556 T721 C635 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383557 T680 C633 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383557 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c505069 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383557 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@533045a7 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383558 T721 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 383558 T680 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 383558 T716 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_4,generation=4}
[junit4:junit4] 2> 383560 T720 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_4,generation=4}
[junit4:junit4] 2> 383560 T716 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 383560 T720 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 383563 T716 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@4b978a7b main
[junit4:junit4] 2> 383563 T720 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@45325271 main
[junit4:junit4] 2> 383564 T716 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383565 T720 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383565 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b978a7b main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 383565 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45325271 main{StandardDirectoryReader(segments_4:11:nrt _2(5.0):c1)}
[junit4:junit4] 2> 383566 T716 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 383566 T720 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 383567 T661 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 29
[junit4:junit4] 2> 383569 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383571 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383575 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383579 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383580 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383584 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383587 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383589 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383590 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383597 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383597 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383601 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326361&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383603 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 383609 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383609 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383614 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326373&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383615 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 383616 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383623 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383624 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383628 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326387&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383629 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=9
[junit4:junit4] 2> 383637 T708 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383637 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=1
[junit4:junit4] 2> 383640 T708 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326400&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383641 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383647 T608 C636 P14091 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383653 T608 C636 P14091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@162de166 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1100d40e),segFN=segments_5,generation=5}
[junit4:junit4] 2> 383654 T608 C636 P14091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 383656 T608 C636 P14091 oass.SolrIndexSearcher.<init> Opening Searcher@578f989f main
[junit4:junit4] 2> 383657 T608 C636 P14091 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383658 T622 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@578f989f main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 383659 T608 C636 P14091 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1 (1439091210491789312)],commit=} 0 14
[junit4:junit4] 2> 383671 T660 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:24273/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439091210512760832)]} 0 2
[junit4:junit4] 2> 383672 T723 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:20834/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1439091210512760832)]} 0 7
[junit4:junit4] 2> 383673 T678 C633 P20834 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:24273/collection1/, StdNode: http://127.0.0.1:35832/collection1/, StdNode: http://127.0.0.1:17310/collection1/, StdNode: http://127.0.0.1:20834/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 383677 T723 C632 P24273 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383677 T728 C635 P17310 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383677 T727 C634 P35832 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383677 T681 C633 P20834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 383678 T728 C635 P17310 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383679 T681 C633 P20834 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 383680 T728 C635 P17310 oass.SolrIndexSearcher.<init> Opening Searcher@5608e1d9 main
[junit4:junit4] 2> 383680 T681 C633 P20834 oass.SolrIndexSearcher.<init> Opening Searcher@80e140a main
[junit4:junit4] 2> 383681 T728 C635 P17310 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383681 T681 C633 P20834 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383682 T656 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5608e1d9 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383682 T690 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@80e140a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 383683 T728 C635 P17310 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 383683 T723 C632 P24273 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@d79a152 lockFactory=org.apache.lucene.store.NativeFSLockFactory@40496596),segFN=segments_5,generation=5}
[junit4:junit4] 2> 383683 T681 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 6
[junit4:junit4] 2> 383684 T723 C632 P24273 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 383684 T727 C634 P35832 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@50963028 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6125f991),segFN=segments_5,generation=5}
[junit4:junit4] 2> 383685 T727 C634 P35832 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 383687 T723 C632 P24273 oass.SolrIndexSearcher.<init> Opening Searcher@31bff073 main
[junit4:junit4] 2> 383688 T723 C632 P24273 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383688 T727 C634 P35832 oass.SolrIndexSearcher.<init> Opening Searcher@7886d870 main
[junit4:junit4] 2> 383689 T640 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31bff073 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 383690 T727 C634 P35832 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 383690 T723 C632 P24273 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> 383690 T672 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7886d870 main{StandardDirectoryReader(segments_5:15:nrt _3(5.0):c1)}
[junit4:junit4] 2> 383691 T727 C634 P35832 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 383692 T678 C633 P20834 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={commit=true&wt=javabin&version=2} {add=[1],commit=} 0 30
[junit4:junit4] 2> 383694 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383696 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383701 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383705 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=1 status=0 QTime=1
[junit4:junit4] 2> 383706 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383710 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383713 T678 C633 P20834 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&distrib=false&wt=javabin&version=2&sanity_check=non_distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383715 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383717 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383723 T650 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383723 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383727 T628 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326487&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383728 T633 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383736 T644 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383736 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383741 T630 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326499&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383742 T661 C634 P35832 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=10
[junit4:junit4] 2> 383744 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 383750 T646 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=0 status=0 QTime=0
[junit4:junit4] 2> 383750 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true&version=2&rows=10&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=0
[junit4:junit4] 2> 383754 T705 C632 P24273 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326514&shard.url=127.0.0.1:24273/collection1/|127.0.0.1:35832/collection1/&ids=1&q=id:1&distrib=false&wt=javabin&isShard=true&version=2&sanity_check=distrib_id_1_lookup} status=0 QTime=0
[junit4:junit4] 2> 383755 T649 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=id:1&wt=javabin&version=2&sanity_check=distrib_id_1_lookup} hits=1 status=0 QTime=8
[junit4:junit4] 2> 383761 T709 C635 P17310 oasc.SolrCore.execute [collection1] webapp= path=/select params={NOW=1372424326525&shard.url=127.0.0.1:17310/collection1/|127.0.0.1:20834/collection1/&fl=id,score&start=0&q=id:1&distrib=false&wt=javabin&isShard=true&fsv=true
[...truncated too long message...]
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {foo_b=Lucene41(blocksize=128), n_f1=PostingsFormat(name=Direct), cat=PostingsFormat(name=MockSep), foo_d=MockVariableIntBlock(baseBlockSize=81), n_tl1=PostingsFormat(name=Direct), foo_f=Lucene41(blocksize=128), n_d1=PostingsFormat(name=MockSep), rnd_b=MockVariableIntBlock(baseBlockSize=81), intDefault=PostingsFormat(name=MockSep), n_td1=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=MockSep), id=MockVariableIntBlock(baseBlockSize=81), range_facet_sl=Lucene41(blocksize=128), range_facet_si=PostingsFormat(name=Direct), oddField_s=Lucene41(blocksize=128), sequence_i=Lucene41(blocksize=128), name=MockVariableIntBlock(baseBlockSize=81), foo_i=PostingsFormat(name=Direct), regex_dup_B_s=PostingsFormat(name=MockSep), multiDefault=PostingsFormat(name=Direct), n_tf1=PostingsFormat(name=MockSep), n_dt1=PostingsFormat(name=Direct), genre_s=Lucene41(blocksize=128), author_t=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=MockSep), n_ti1=MockVariableIntBlock(baseBlockSize=81), text=Lucene41(blocksize=128), _version_=PostingsFormat(name=MockSep), val_i=MockVariableIntBlock(baseBlockSize=81), SubjectTerms_mfacet=PostingsFormat(name=MockSep), series_t=PostingsFormat(name=Direct), a_t=PostingsFormat(name=MockSep), n_tdt1=PostingsFormat(name=MockSep), regex_dup_A_s=Lucene41(blocksize=128), price=Lucene41(blocksize=128), other_tl1=PostingsFormat(name=MockSep), n_l1=PostingsFormat(name=MockSep), a_si=PostingsFormat(name=Direct), inStock=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=es_US, timezone=America/Guyana
[junit4:junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=16,threads=2,free=96278400,total=336592896
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestDocSet, CSVRequestHandlerTest, TestShardHandlerFactory, TestLRUCache, TestRandomDVFaceting, ReturnFieldsTest, StatelessScriptUpdateProcessorFactoryTest, SliceStateTest, TestSweetSpotSimilarityFactory, PrimUtilsTest, ChaosMonkeyNothingIsSafeTest, TestQuerySenderListener, TestManagedSchemaFieldResource, OverseerCollectionProcessorTest, ResponseLogComponentTest, TestPluginEnable, TestCoreContainer, ClusterStateUpdateTest, ParsingFieldUpdateProcessorsTest, BasicDistributedZkTest]
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZkTest -Dtests.seed=C78A344FD79A7AD2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=America/Guyana -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR 0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.BasicDistributedZkTest:
[junit4:junit4] > 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4:junit4] > 1) Thread[id=1054, name=recoveryCmdExecutor-343-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4:junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4:junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4:junit4] > at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:297)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([C78A344FD79A7AD2]:0)
[junit4:junit4] Completed on J0 in 217.75s, 1 test, 2 errors <<< FAILURES!
[...truncated 779 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:386: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:366: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:190: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:443: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1245: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 suite-level errors, 849 ignored (5 assumptions)
Total time: 38 minutes 30 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure