You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/21 20:23:25 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 583 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/583/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
All tests passed
Build Log:
[...truncated 9420 lines...]
[junit4:junit4] ERROR: JVM J0 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/jre/bin/java -XX:+UseCompressedOops -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=2A5CD82925284E6F -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp -Dclover.db.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Dfile.encoding=ISO-8859-1 -classpath /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/test:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-test-framework/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/test-framework/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-solrj/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/codecs/lucene-codecs-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/cglib-nodep-2.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-cli-1.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-codec-1.7.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-fileupload-1.2.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/commons-lang-2.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/easymock-3.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/guava-14.0.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/javax.servlet-api-3.0.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/objenesis-1.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet-2.1.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/core/lib/spatial4j-0.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/commons-io-2.1.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpclient-4.2.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpcore-4.2.2.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/httpmime-4.2.3.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/jcl-over-slf4j-1.6.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/jul-to-slf4j-1.6.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/log4j-1.2.16.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/noggit-0.5.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-api-1.6.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/slf4j-log4j12-1.6.6.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/wstx-asl-3.2.7.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/solrj/lib/zookeeper-3.4.5.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-continuation-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-deploy-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-http-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-io-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-jmx-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-security-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-server-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-servlet-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-util-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-webapp-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/jetty-xml-8.1.10.v20130312.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/lib/servlet-api-3.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/core/classes/java:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/junit-4.10.jar:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/test-framework/lib/randomizedtesting-runner-2.0.10.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/Users/jenkins/.ant/lib/ivy-2.3.0.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/Users/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/Library/Java/JavaVirtualMachines/jdk1.7.0_21.jdk/Contents/Home/lib/tools.jar:/Users/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.10.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -flush -eventsfile /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20130621_175959_367.events @/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20130621_175959_367.suites
[junit4:junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash. Process output piped in logs above.
[junit4:junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1254)
[junit4:junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:67)
[junit4:junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:822)
[junit4:junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:819)
[junit4:junit4] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] at java.lang.Thread.run(Thread.java:722)
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash. Process output piped in logs above.
Total time: 65 minutes 6 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 587 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/587/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:50917 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:50917 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([ED5D37BBB19952BE:6CBBB9A3C6C63282]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:51169/collection1lastClient and got 259 from http://127.0.0.1:51176/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51169/collection1lastClient and got 259 from http://127.0.0.1:51176/collection1
at __randomizedtesting.SeedInfo.seed([ED5D37BBB19952BE:6CBBB9A3C6C63282]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9517 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4:junit4] 2> 1574660 T3497 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1574675 T3497 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ShardSplitTest-1371936056644
[junit4:junit4] 2> 1574678 T3497 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1574680 T3498 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1574780 T3497 oasc.ZkTestServer.run start zk server on port:50910
[junit4:junit4] 2> 1574791 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1574802 T3504 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c4378b4 name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1574803 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1574804 T3497 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1574815 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1574822 T3506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6af31e19 name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1574822 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1574823 T3497 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1574833 T3497 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1574842 T3497 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1574849 T3497 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1574859 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1574860 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1574880 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1574881 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1574892 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1574894 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1574903 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1574905 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1574919 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1574921 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1574930 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1574931 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1574944 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1574946 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1574952 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1574953 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1574961 T3497 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1574962 T3497 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1575432 T3497 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1575472 T3497 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50913
[junit4:junit4] 2> 1575473 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1575474 T3497 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1575479 T3497 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938
[junit4:junit4] 2> 1575480 T3497 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/solr.xml
[junit4:junit4] 2> 1575481 T3497 oasc.CoreContainer.<init> New CoreContainer 700978093
[junit4:junit4] 2> 1575481 T3497 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/'
[junit4:junit4] 2> 1575482 T3497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/'
[junit4:junit4] 2> 1575674 T3497 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1575676 T3497 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1575677 T3497 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1575678 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1575680 T3497 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1575681 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1575682 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1575683 T3497 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1575684 T3497 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1575685 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1575716 T3497 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1575719 T3497 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50910/solr
[junit4:junit4] 2> 1575720 T3497 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1575726 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1575729 T3517 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5daf5690 name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1575729 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1575734 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1575758 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1575763 T3519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fce8e75 name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1575764 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1575768 T3497 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1575775 T3497 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1575782 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1575787 T3497 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50913_
[junit4:junit4] 2> 1575789 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50913_
[junit4:junit4] 2> 1575796 T3497 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1575826 T3497 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1575833 T3497 oasc.Overseer.start Overseer (id=89911201408876547-127.0.0.1:50913_-n_0000000000) starting
[junit4:junit4] 2> 1575840 T3497 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1575850 T3521 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1575852 T3520 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1575857 T3497 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1575863 T3497 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1575868 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1575883 T3522 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1575884 T3522 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1577361 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1577363 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50913_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50913"}
[junit4:junit4] 2> 1577363 T3520 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1577363 T3520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1577371 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1577892 T3522 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/collection1
[junit4:junit4] 2> 1577892 T3522 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1577894 T3522 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1577894 T3522 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1577898 T3522 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/collection1/'
[junit4:junit4] 2> 1577899 T3522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1577900 T3522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/collection1/lib/README' to classloader
[junit4:junit4] 2> 1578021 T3522 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1578151 T3522 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1578155 T3522 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1578161 T3522 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1579286 T3522 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1579298 T3522 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1579301 T3522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1579309 T3522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1579374 T3522 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1579375 T3522 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-controljetty-1371936056938/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data/
[junit4:junit4] 2> 1579375 T3522 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@620533a5
[junit4:junit4] 2> 1579376 T3522 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1579377 T3522 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data
[junit4:junit4] 2> 1579378 T3522 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data/index/
[junit4:junit4] 2> 1579378 T3522 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1579379 T3522 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data/index
[junit4:junit4] 2> 1579388 T3522 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55dd8d7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@27a9f213),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1579389 T3522 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1579397 T3522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1579398 T3522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1579401 T3522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1579403 T3522 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1579404 T3522 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1579405 T3522 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1579406 T3522 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1579407 T3522 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1579408 T3522 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1579430 T3522 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1579449 T3522 oass.SolrIndexSearcher.<init> Opening Searcher@3201cb2d main
[junit4:junit4] 2> 1579451 T3522 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/control/data/tlog
[junit4:junit4] 2> 1579454 T3522 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1579455 T3522 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1579463 T3523 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3201cb2d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1579468 T3522 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1579468 T3522 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50913 collection:control_collection shard:shard1
[junit4:junit4] 2> 1579472 T3522 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1579493 T3522 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1579501 T3522 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1579502 T3522 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1579503 T3522 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50913/collection1/
[junit4:junit4] 2> 1579503 T3522 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1579504 T3522 oasc.SyncStrategy.syncToMe http://127.0.0.1:50913/collection1/ has no replicas
[junit4:junit4] 2> 1579504 T3522 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50913/collection1/
[junit4:junit4] 2> 1579505 T3522 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1580414 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1580440 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1580461 T3522 oasc.ZkController.register We are http://127.0.0.1:50913/collection1/ and leader is http://127.0.0.1:50913/collection1/
[junit4:junit4] 2> 1580462 T3522 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50913
[junit4:junit4] 2> 1580462 T3522 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1580462 T3522 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1580463 T3522 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1580467 T3522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1580470 T3497 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1580471 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1580472 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1580497 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1580509 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1580512 T3526 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@142b1f6a name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1580513 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1580523 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1580535 T3497 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1581086 T3497 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1581096 T3497 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50917
[junit4:junit4] 2> 1581097 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1581098 T3497 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1581098 T3497 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503
[junit4:junit4] 2> 1581099 T3497 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/solr.xml
[junit4:junit4] 2> 1581099 T3497 oasc.CoreContainer.<init> New CoreContainer 1100866948
[junit4:junit4] 2> 1581100 T3497 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/'
[junit4:junit4] 2> 1581101 T3497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/'
[junit4:junit4] 2> 1581316 T3497 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1581322 T3497 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1581323 T3497 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1581324 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1581324 T3497 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1581325 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1581325 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1581326 T3497 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1581327 T3497 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1581328 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1581348 T3497 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1581349 T3497 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50910/solr
[junit4:junit4] 2> 1581350 T3497 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1581352 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1581357 T3537 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@212187d4 name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1581358 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1581363 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1581381 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1581391 T3539 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@617b3521 name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1581392 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1581403 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1581958 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1581961 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50913_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50913"}
[junit4:junit4] 2> 1581978 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1581979 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1581979 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1582415 T3497 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50917_
[junit4:junit4] 2> 1582419 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50917_
[junit4:junit4] 2> 1582427 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1582428 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1582433 T3539 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1582433 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1582443 T3519 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1582444 T3526 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1582470 T3540 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1582474 T3540 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1583506 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1583508 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50917_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50917"}
[junit4:junit4] 2> 1583509 T3520 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 1583510 T3520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1583523 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1583524 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1583525 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1584481 T3540 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/collection1
[junit4:junit4] 2> 1584482 T3540 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1584484 T3540 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1584484 T3540 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1584488 T3540 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/collection1/'
[junit4:junit4] 2> 1584492 T3540 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1584493 T3540 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/collection1/lib/README' to classloader
[junit4:junit4] 2> 1584630 T3540 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1584771 T3540 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1584776 T3540 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1584787 T3540 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1586236 T3540 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1586243 T3540 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1586248 T3540 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1586271 T3540 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1586374 T3540 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1586376 T3540 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty1-1371936062503/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1/
[junit4:junit4] 2> 1586376 T3540 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@620533a5
[junit4:junit4] 2> 1586377 T3540 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1586379 T3540 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1
[junit4:junit4] 2> 1586380 T3540 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1/index/
[junit4:junit4] 2> 1586380 T3540 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1586382 T3540 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1/index
[junit4:junit4] 2> 1586389 T3540 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1586390 T3540 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1586402 T3540 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1586404 T3540 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1586407 T3540 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1586410 T3540 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1586412 T3540 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1586413 T3540 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1586415 T3540 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1586418 T3540 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1586420 T3540 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1586449 T3540 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1586470 T3540 oass.SolrIndexSearcher.<init> Opening Searcher@5da5cccf main
[junit4:junit4] 2> 1586471 T3540 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty1/tlog
[junit4:junit4] 2> 1586472 T3540 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1586473 T3540 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1586483 T3541 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5da5cccf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1586485 T3540 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1586485 T3540 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50917 collection:collection1 shard:shard1
[junit4:junit4] 2> 1586487 T3540 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1586507 T3540 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1586516 T3540 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1586517 T3540 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1586518 T3540 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50917/collection1/
[junit4:junit4] 2> 1586518 T3540 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1586519 T3540 oasc.SyncStrategy.syncToMe http://127.0.0.1:50917/collection1/ has no replicas
[junit4:junit4] 2> 1586520 T3540 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50917/collection1/
[junit4:junit4] 2> 1586520 T3540 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1586552 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1586584 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1586585 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1586584 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1586594 T3540 oasc.ZkController.register We are http://127.0.0.1:50917/collection1/ and leader is http://127.0.0.1:50917/collection1/
[junit4:junit4] 2> 1586596 T3540 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50917
[junit4:junit4] 2> 1586596 T3540 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1586597 T3540 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1586598 T3540 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1586604 T3540 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1586610 T3497 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1586612 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1586613 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1587166 T3497 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1587195 T3497 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50920
[junit4:junit4] 2> 1587196 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1587197 T3497 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1587198 T3497 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612
[junit4:junit4] 2> 1587198 T3497 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/solr.xml
[junit4:junit4] 2> 1587199 T3497 oasc.CoreContainer.<init> New CoreContainer 1780329248
[junit4:junit4] 2> 1587200 T3497 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/'
[junit4:junit4] 2> 1587200 T3497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/'
[junit4:junit4] 2> 1587455 T3497 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1587456 T3497 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1587458 T3497 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1587459 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1587461 T3497 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1587462 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1587463 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1587464 T3497 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1587465 T3497 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1587466 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1587501 T3497 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1587502 T3497 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50910/solr
[junit4:junit4] 2> 1587504 T3497 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1587508 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1587515 T3553 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c3aa588 name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1587520 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1587526 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1587547 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1587559 T3555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66f6a4c name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1587562 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1587575 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1588119 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1588122 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50917_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50917"}
[junit4:junit4] 2> 1588136 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588137 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588138 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588140 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588586 T3497 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50920_
[junit4:junit4] 2> 1588589 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50920_
[junit4:junit4] 2> 1588596 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588596 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1588604 T3539 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1588605 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1588607 T3555 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1588608 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1588613 T3526 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1588616 T3519 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1588634 T3556 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1588635 T3556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1589659 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1589662 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50920_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50920"}
[junit4:junit4] 2> 1589663 T3520 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1589663 T3520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1589676 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1589682 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1589682 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1589683 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1590643 T3556 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/collection1
[junit4:junit4] 2> 1590644 T3556 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1590646 T3556 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1590647 T3556 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1590652 T3556 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/collection1/'
[junit4:junit4] 2> 1590655 T3556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1590656 T3556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/collection1/lib/README' to classloader
[junit4:junit4] 2> 1590772 T3556 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1591034 T3556 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1591038 T3556 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1591052 T3556 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1592239 T3556 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1592251 T3556 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1592257 T3556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1592267 T3556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1592344 T3556 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1592345 T3556 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty2-1371936068612/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2/
[junit4:junit4] 2> 1592345 T3556 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@620533a5
[junit4:junit4] 2> 1592346 T3556 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1592348 T3556 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2
[junit4:junit4] 2> 1592348 T3556 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2/index/
[junit4:junit4] 2> 1592349 T3556 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1592350 T3556 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2/index
[junit4:junit4] 2> 1592356 T3556 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1592356 T3556 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1592365 T3556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1592366 T3556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1592369 T3556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1592371 T3556 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1592372 T3556 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1592372 T3556 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1592373 T3556 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1592375 T3556 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1592376 T3556 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1592400 T3556 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1592415 T3556 oass.SolrIndexSearcher.<init> Opening Searcher@2de8a5c9 main
[junit4:junit4] 2> 1592416 T3556 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty2/tlog
[junit4:junit4] 2> 1592417 T3556 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1592418 T3556 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1592426 T3557 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2de8a5c9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1592431 T3556 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1592432 T3556 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50920 collection:collection1 shard:shard2
[junit4:junit4] 2> 1592433 T3556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1592451 T3556 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1592458 T3556 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1592459 T3556 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1592459 T3556 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50920/collection1/
[junit4:junit4] 2> 1592460 T3556 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1592460 T3556 oasc.SyncStrategy.syncToMe http://127.0.0.1:50920/collection1/ has no replicas
[junit4:junit4] 2> 1592460 T3556 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50920/collection1/
[junit4:junit4] 2> 1592461 T3556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1592722 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1592742 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1592742 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1592742 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1592742 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1592787 T3556 oasc.ZkController.register We are http://127.0.0.1:50920/collection1/ and leader is http://127.0.0.1:50920/collection1/
[junit4:junit4] 2> 1592788 T3556 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50920
[junit4:junit4] 2> 1592788 T3556 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1592789 T3556 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1592789 T3556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1592793 T3556 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1592796 T3497 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1592798 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1592798 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1593382 T3497 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1593397 T3497 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50923
[junit4:junit4] 2> 1593405 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1593406 T3497 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1593407 T3497 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787
[junit4:junit4] 2> 1593408 T3497 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/solr.xml
[junit4:junit4] 2> 1593409 T3497 oasc.CoreContainer.<init> New CoreContainer 1413083806
[junit4:junit4] 2> 1593410 T3497 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/'
[junit4:junit4] 2> 1593412 T3497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/'
[junit4:junit4] 2> 1593656 T3497 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1593657 T3497 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1593658 T3497 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1593659 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1593661 T3497 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1593661 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1593662 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1593663 T3497 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1593663 T3497 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1593664 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1593684 T3497 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1593685 T3497 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50910/solr
[junit4:junit4] 2> 1593686 T3497 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1593690 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1593694 T3569 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b0b8cfd name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1593695 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1593702 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1593730 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1593736 T3571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7faf206a name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1593737 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1593751 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1594263 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1594267 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50920_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50920"}
[junit4:junit4] 2> 1594286 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594286 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594286 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594287 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594287 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594768 T3497 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50923_
[junit4:junit4] 2> 1594772 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50923_
[junit4:junit4] 2> 1594780 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594781 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1594784 T3571 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1594785 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1594786 T3555 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1594787 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1594788 T3539 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1594789 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1594793 T3526 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1594796 T3519 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1594818 T3572 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1594818 T3572 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1595804 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1595807 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50923_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50923"}
[junit4:junit4] 2> 1595808 T3520 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1595808 T3520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1595831 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1595833 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1595833 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1595834 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1595833 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1596824 T3572 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/collection1
[junit4:junit4] 2> 1596825 T3572 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1596826 T3572 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1596827 T3572 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1596830 T3572 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/collection1/'
[junit4:junit4] 2> 1596831 T3572 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1596832 T3572 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/collection1/lib/README' to classloader
[junit4:junit4] 2> 1596931 T3572 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1597015 T3572 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1597018 T3572 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1597029 T3572 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1598220 T3572 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1598224 T3572 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1598229 T3572 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1598240 T3572 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1598316 T3572 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1598317 T3572 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty3-1371936074787/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/
[junit4:junit4] 2> 1598318 T3572 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@620533a5
[junit4:junit4] 2> 1598319 T3572 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1598322 T3572 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3
[junit4:junit4] 2> 1598322 T3572 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index/
[junit4:junit4] 2> 1598323 T3572 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1598324 T3572 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index
[junit4:junit4] 2> 1598329 T3572 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227155c3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@601c2328),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1598330 T3572 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1598337 T3572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1598338 T3572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1598340 T3572 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1598342 T3572 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1598347 T3572 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1598349 T3572 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1598350 T3572 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1598353 T3572 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1598355 T3572 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1598382 T3572 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1598393 T3572 oass.SolrIndexSearcher.<init> Opening Searcher@4b1c1dce main
[junit4:junit4] 2> 1598395 T3572 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/tlog
[junit4:junit4] 2> 1598396 T3572 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1598397 T3572 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1598413 T3573 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b1c1dce main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1598418 T3572 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1598419 T3572 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50923 collection:collection1 shard:shard1
[junit4:junit4] 2> 1598428 T3572 oasc.ZkController.register We are http://127.0.0.1:50923/collection1/ and leader is http://127.0.0.1:50917/collection1/
[junit4:junit4] 2> 1598429 T3572 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50923
[junit4:junit4] 2> 1598429 T3572 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1598430 T3572 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C617 name=collection1 org.apache.solr.core.SolrCore@4328c4c0 url=http://127.0.0.1:50923/collection1 node=127.0.0.1:50923_ C617_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50923_, base_url=http://127.0.0.1:50923}
[junit4:junit4] 2> 1598431 T3574 C617 P50923 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1598432 T3572 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1598433 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1598434 T3574 C617 P50923 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1598435 T3574 C617 P50923 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1598436 T3497 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1598437 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1598439 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1598451 T3574 C617 P50923 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1598480 T3535 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1599004 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1599009 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50923_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50923"}
[junit4:junit4] 2> 1599053 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1599054 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1599055 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1599054 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1599054 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1599158 T3497 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1599183 T3497 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50927
[junit4:junit4] 2> 1599185 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1599187 T3497 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1599188 T3497 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430
[junit4:junit4] 2> 1599189 T3497 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/solr.xml
[junit4:junit4] 2> 1599190 T3497 oasc.CoreContainer.<init> New CoreContainer 1776469313
[junit4:junit4] 2> 1599192 T3497 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/'
[junit4:junit4] 2> 1599194 T3497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/'
[junit4:junit4] 2> 1599482 T3535 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1599485 T3535 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&nodeName=127.0.0.1:50923_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1005
[junit4:junit4] 2> 1599505 T3497 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1599506 T3497 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1599507 T3497 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1599508 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1599510 T3497 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1599513 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1599515 T3497 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1599517 T3497 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1599520 T3497 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1599522 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1599553 T3497 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1599554 T3497 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50910/solr
[junit4:junit4] 2> 1599556 T3497 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1599562 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1599575 T3586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a25462d name:ZooKeeperConnection Watcher:127.0.0.1:50910 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1599578 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1599590 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1599630 T3497 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1599638 T3588 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6919be61 name:ZooKeeperConnection Watcher:127.0.0.1:50910/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1599639 T3497 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1599649 T3497 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1600659 T3497 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50927_
[junit4:junit4] 2> 1600662 T3497 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50927_
[junit4:junit4] 2> 1600671 T3571 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600672 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1600672 T3588 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600691 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1600692 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1600704 T3519 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600707 T3555 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600716 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1600715 T3526 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600711 T3539 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1600717 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1600711 T3589 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1600719 T3589 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C617_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50923_, base_url=http://127.0.0.1:50923}
[junit4:junit4] 2> 1601503 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50917/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1601504 T3574 C617 P50923 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50923 START replicas=[http://127.0.0.1:50917/collection1/] nUpdates=100
[junit4:junit4] 2> 1601505 T3574 C617 P50923 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1601506 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1601506 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1601507 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1601507 T3574 C617 P50923 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1601508 T3574 C617 P50923 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50917/collection1/. core=collection1
[junit4:junit4] 2> 1601509 T3574 C617 P50923 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C618 name=collection1 org.apache.solr.core.SolrCore@40f64ab8 url=http://127.0.0.1:50917/collection1 node=127.0.0.1:50917_ C618_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50917_, base_url=http://127.0.0.1:50917, leader=true}
[junit4:junit4] 2> 1601572 T3532 C618 P50917 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1601578 T3531 C618 P50917 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 1601586 T3532 C618 P50917 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1601588 T3532 C618 P50917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1601591 T3532 C618 P50917 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1601592 T3532 C618 P50917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1601593 T3532 C618 P50917 oass.SolrIndexSearcher.<init> Opening Searcher@2ec5f050 realtime
[junit4:junit4] 2> 1601594 T3532 C618 P50917 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1601596 T3532 C618 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 1601599 T3574 C617 P50923 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1601600 T3574 C617 P50923 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1601612 T3534 C618 P50917 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1601614 T3534 C618 P50917 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=7
[junit4:junit4] 2> 1601616 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1601616 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1601617 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1601621 T3530 C618 P50917 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 1601623 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1601626 T3574 C617 P50923 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index.20130622232123591
[junit4:junit4] 2> 1601627 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@78427ca8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1312f95a) fullCopy=false
[junit4:junit4] 2> 1601634 T3535 C618 P50917 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 1601641 T3574 C617 P50923 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1601643 T3574 C617 P50923 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1601644 T3574 C617 P50923 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1601648 T3574 C617 P50923 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227155c3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@601c2328),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227155c3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@601c2328),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1601648 T3574 C617 P50923 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1601649 T3574 C617 P50923 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1601649 T3574 C617 P50923 oass.SolrIndexSearcher.<init> Opening Searcher@2cb0f005 main
[junit4:junit4] 2> 1601653 T3573 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2cb0f005 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1601655 T3574 C617 P50923 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index.20130622232123591 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index.20130622232123591;done=true>>]
[junit4:junit4] 2> 1601656 T3574 C617 P50923 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index.20130622232123591
[junit4:junit4] 2> 1601656 T3574 C617 P50923 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty3/index.20130622232123591
[junit4:junit4] 2> 1601658 T3574 C617 P50923 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1601658 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1601658 T3574 C617 P50923 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1601658 T3574 C617 P50923 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1601661 T3574 C617 P50923 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1602084 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1602086 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50927_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50927"}
[junit4:junit4] 2> 1602086 T3520 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 1602086 T3520 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1602093 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50923_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50923"}
[junit4:junit4] 2> 1602106 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602106 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602107 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602107 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602108 T3588 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602108 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1602729 T3589 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/collection1
[junit4:junit4] 2> 1602730 T3589 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1602731 T3589 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1602732 T3589 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1602734 T3589 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/collection1/'
[junit4:junit4] 2> 1602736 T3589 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1602737 T3589 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/collection1/lib/README' to classloader
[junit4:junit4] 2> 1602841 T3589 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1602935 T3589 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1602938 T3589 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1602946 T3589 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1604066 T3589 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1604078 T3589 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1604086 T3589 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1604098 T3589 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1604204 T3589 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1604212 T3589 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-jetty4-1371936080430/collection1/, dataDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/
[junit4:junit4] 2> 1604219 T3589 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@620533a5
[junit4:junit4] 2> 1604219 T3589 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1604220 T3589 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4
[junit4:junit4] 2> 1604220 T3589 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index/
[junit4:junit4] 2> 1604221 T3589 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1604221 T3589 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index
[junit4:junit4] 2> 1604228 T3589 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@53af6fdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@45d4b35),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1604229 T3589 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1604239 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1604240 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1604241 T3589 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1604243 T3589 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1604244 T3589 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1604245 T3589 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1604248 T3589 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1604249 T3589 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1604250 T3589 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1604266 T3589 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1604277 T3589 oass.SolrIndexSearcher.<init> Opening Searcher@634969c6 main
[junit4:junit4] 2> 1604278 T3589 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/tlog
[junit4:junit4] 2> 1604279 T3589 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1604280 T3589 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1604290 T3592 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@634969c6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1604294 T3589 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1604295 T3589 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50927 collection:collection1 shard:shard2
[junit4:junit4] 2> 1604302 T3589 oasc.ZkController.register We are http://127.0.0.1:50927/collection1/ and leader is http://127.0.0.1:50920/collection1/
[junit4:junit4] 2> 1604303 T3589 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50927
[junit4:junit4] 2> 1604304 T3589 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1604304 T3589 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C619 name=collection1 org.apache.solr.core.SolrCore@6ed4275c url=http://127.0.0.1:50927/collection1 node=127.0.0.1:50927_ C619_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50927_, base_url=http://127.0.0.1:50927}
[junit4:junit4] 2> 1604305 T3593 C619 P50927 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1604306 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1604307 T3593 C619 P50927 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1604308 T3593 C619 P50927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1604308 T3589 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1604312 T3593 C619 P50927 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1604313 T3497 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1604314 T3497 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1604314 T3497 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1604341 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1604342 T3551 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1604350 T3497 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1604354 T3497 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1604356 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1605232 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1605235 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50927_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50927"}
[junit4:junit4] 2> 1605248 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605249 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605250 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605260 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605261 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605261 T3588 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1605346 T3551 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1605348 T3551 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:50927_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1006
[junit4:junit4] 2> 1605361 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1606366 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C619_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50927_, base_url=http://127.0.0.1:50927}
[junit4:junit4] 2> 1607350 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50920/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1607351 T3593 C619 P50927 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50927 START replicas=[http://127.0.0.1:50920/collection1/] nUpdates=100
[junit4:junit4] 2> 1607352 T3593 C619 P50927 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1607353 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1607353 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1607353 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1607354 T3593 C619 P50927 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1607354 T3593 C619 P50927 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50920/collection1/. core=collection1
[junit4:junit4] 2> 1607355 T3593 C619 P50927 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C620 name=collection1 org.apache.solr.core.SolrCore@4a685555 url=http://127.0.0.1:50920/collection1 node=127.0.0.1:50920_ C620_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50920_, base_url=http://127.0.0.1:50920, leader=true}
[junit4:junit4] 2> 1607410 T3548 C620 P50920 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 1607420 T3549 C620 P50920 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1607429 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1607435 T3549 C620 P50920 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1607436 T3549 C620 P50920 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1607438 T3549 C620 P50920 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1607438 T3549 C620 P50920 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1607439 T3549 C620 P50920 oass.SolrIndexSearcher.<init> Opening Searcher@5171275e realtime
[junit4:junit4] 2> 1607440 T3549 C620 P50920 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1607441 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 1607443 T3593 C619 P50927 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1607444 T3593 C619 P50927 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1607452 T3546 C620 P50920 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1607453 T3546 C620 P50920 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3
[junit4:junit4] 2> 1607455 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1607456 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1607457 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1607461 T3547 C620 P50920 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 1607464 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1607469 T3593 C619 P50927 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index.20130622232129432
[junit4:junit4] 2> 1607470 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@4b071e83 lockFactory=org.apache.lucene.store.NativeFSLockFactory@9dee16c) fullCopy=false
[junit4:junit4] 2> 1607475 T3551 C620 P50920 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 1607479 T3593 C619 P50927 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1607482 T3593 C619 P50927 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1607483 T3593 C619 P50927 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1607486 T3593 C619 P50927 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@53af6fdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@45d4b35),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@53af6fdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@45d4b35),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1607488 T3593 C619 P50927 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1607489 T3593 C619 P50927 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1607490 T3593 C619 P50927 oass.SolrIndexSearcher.<init> Opening Searcher@53c025da main
[junit4:junit4] 2> 1607493 T3592 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53c025da main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1607495 T3593 C619 P50927 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index.20130622232129432 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index.20130622232129432;done=true>>]
[junit4:junit4] 2> 1607495 T3593 C619 P50927 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index.20130622232129432
[junit4:junit4] 2> 1607496 T3593 C619 P50927 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ShardSplitTest-1371936056643/jetty4/index.20130622232129432
[junit4:junit4] 2> 1607496 T3593 C619 P50927 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1607496 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1607497 T3593 C619 P50927 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1607497 T3593 C619 P50927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1607501 T3593 C619 P50927 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1608283 T3520 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1608285 T3520 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50927_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50927"}
[junit4:junit4] 2> 1608297 T3526 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608298 T3588 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608306 T3571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608308 T3539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608308 T3519 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608309 T3555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1608436 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1608439 T3497 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C621 name=collection1 org.apache.solr.core.SolrCore@2641f2dc url=http://127.0.0.1:50913/collection1 node=127.0.0.1:50913_ C621_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:50913_, base_url=http://127.0.0.1:50913, leader=true}
[junit4:junit4] 2> 1608458 T3513 C621 P50913 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1608469 T3513 C621 P50913 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55dd8d7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@27a9f213),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1608470 T3513 C621 P50913 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1608477 T3513 C621 P50913 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55dd8d7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@27a9f213),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@55dd8d7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@27a9f213),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1608479 T3513 C621 P50913 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1608481 T3513 C621 P50913 oass.SolrIndexSearcher.<init> Opening Searcher@1943391f main
[junit4:junit4] 2> 1608483 T3513 C621 P50913 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1608488 T3523 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1943391f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1608497 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 40
[junit4:junit4] 2> C619_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50927_, base_url=http://127.0.0.1:50927}
[junit4:junit4] 2> 1608528 T3584 C619 P50927 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50917/collection1/, StdNode: http://127.0.0.1:50923/collection1/, StdNode: http://127.0.0.1:50920/collection1/, StdNode: http://127.0.0.1:50927/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C622 name=collection1 org.apache.solr.core.SolrCore@40f64ab8 url=http://127.0.0.1:50917/collection1 node=127.0.0.1:50917_ C622_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50917_, base_url=http://127.0.0.1:50917, leader=true}
[junit4:junit4] 2> 1608551 T3530 C622 P50917 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1608566 T3550 C620 P50920 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C623 name=collection1 org.apache.solr.core.SolrCore@4328c4c0 url=http://127.0.0.1:50923/collection1 node=127.0.0.1:50923_ C623_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50923_, base_url=http://127.0.0.1:50923}
[junit4:junit4] 2> 1608562 T3567 C623 P50923 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1608572 T3550 C620 P50920 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5675db24 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6133671f),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1608572 T3530 C622 P50917 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a946aac lockFactory=org.apache.lucene.store.NativeFSLockFactory@20f63bea),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1608567 T3579 C619 P50927 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1608577 T3530 C622 P50917 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1608576 T3550 C620 P50920 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1608582 T3550 C620 P50920 oass.SolrIndexSearcher.<init> Opening Searcher@125c786a main
[junit4:junit4] 2> 1608584 T3550 C620 P50920 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1608575 T3567 C623 P50923 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227155c3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@601c2328),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@227155c3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@601c2328),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1608581 T3530 C622 P50917 oass.SolrIndexSearcher.<init> Opening Searcher@1ed7caff main
[junit4:junit4] 2> 1608579 T3579 C619 P50927 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@53af6fdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@45d4b35),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@53af6fdc lockFactory=org.apache.lucene.store.NativeFSLockFactory@45d4b35),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1608587 T3579 C619 P50927 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1608586 T3557 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@125c786a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1608590 T3579 C619 P50927 oass.SolrIndexSearcher.<init> Opening Searcher@57442ef1 main
[junit4:junit4] 2> 1608589 T3567 C623 P50923 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1608588 T3530 C622 P50917 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1608588 T3541 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ed7caff main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1608601 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 50
[junit4:junit4] 2> 1608599 T3567 C623 P50923 oass.SolrIndexSearcher.<init> Opening Searcher@72c35eec main
[junit4:junit4] 2> 1608596 T3592 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57442ef1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1608593 T3579 C619 P50927 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1608607 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 41
[junit4:junit4] 2> 1608592 T3550 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 1608607 T3567 C623 P50923 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1608607 T3573 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72c35eec main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1608613 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 51
[junit4:junit4] 2> 1608614 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 86
[junit4:junit4] 2> 1608619 T3497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1608634 T3535 C622 P50917 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1608642 T3563 C623 P50923 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1608647 T3546 C620 P50920 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1608653 T3580 C619 P50927 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1610667 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438579260268740608)} 0 2
[junit4:junit4] 2> 1610688 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438579260276080640&update.from=http://127.0.0.1:50920/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438579260276080640)} 0 2
[junit4:junit4] 2> 1610701 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438579260292857856&update.from=http://127.0.0.1:50917/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438579260292857856)} 0 2
[junit4:junit4] 2> 1610703 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438579260292857856)} 0 14
[junit4:junit4] 2> 1610704 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438579260276080640)} 0 32
[junit4:junit4] 2> 1610712 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0 (1438579260314877952)]} 0 2
[junit4:junit4] 2> 1610734 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!0 (1438579260329558016)]} 0 1
[junit4:junit4] 2> 1610736 T3547 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!0 (1438579260329558016)]} 0 12
[junit4:junit4] 2> 1610737 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!0]} 0 20
[junit4:junit4] 2> 1610747 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1 (1438579260352626688)]} 0 2
[junit4:junit4] 2> 1610759 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!1 (1438579260358918144)]} 0 2
[junit4:junit4] 2> 1610762 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!1 (1438579260358918144)]} 0 10
[junit4:junit4] 2> 1610770 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2 (1438579260376743936)]} 0 2
[junit4:junit4] 2> 1610784 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!2 (1438579260388278272)]} 0 0
[junit4:junit4] 2> 1610786 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!2 (1438579260388278272)]} 0 6
[junit4:junit4] 2> 1610789 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!2]} 0 14
[junit4:junit4] 2> 1610794 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3 (1438579260401909760)]} 0 1
[junit4:junit4] 2> 1610816 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!3 (1438579260419735552)]} 0 1
[junit4:junit4] 2> 1610818 T3551 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[d!3 (1438579260419735552)]} 0 8
[junit4:junit4] 2> 1610820 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!3]} 0 17
[junit4:junit4] 2> 1610825 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4 (1438579260434415616)]} 0 1
[junit4:junit4] 2> 1610837 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!4 (1438579260443852800)]} 0 1
[junit4:junit4] 2> 1610839 T3548 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!4 (1438579260443852800)]} 0 7
[junit4:junit4] 2> 1610840 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!4]} 0 11
[junit4:junit4] 2> 1610845 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5 (1438579260455387136)]} 0 1
[junit4:junit4] 2> 1610857 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!5 (1438579260464824320)]} 0 1
[junit4:junit4] 2> 1610859 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!5 (1438579260464824320)]} 0 6
[junit4:junit4] 2> 1610860 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!5]} 0 11
[junit4:junit4] 2> 1610873 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6 (1438579260482650112)]} 0 7
[junit4:junit4] 2> 1610890 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!6 (1438579260498378752)]} 0 1
[junit4:junit4] 2> 1610893 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!6 (1438579260498378752)]} 0 9
[junit4:junit4] 2> 1610895 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!6]} 0 15
[junit4:junit4] 2> 1610900 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7 (1438579260513058816)]} 0 1
[junit4:junit4] 2> 1610917 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!7 (1438579260525641728)]} 0 1
[junit4:junit4] 2> 1610920 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[h!7 (1438579260525641728)]} 0 9
[junit4:junit4] 2> 1610922 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!7]} 0 15
[junit4:junit4] 2> 1610929 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8 (1438579260543467520)]} 0 1
[junit4:junit4] 2> 1610958 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!8 (1438579260564439040)]} 0 1
[junit4:junit4] 2> 1610960 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[i!8 (1438579260564439040)]} 0 12
[junit4:junit4] 2> 1610962 T3550 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!8]} 0 25
[junit4:junit4] 2> 1610967 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9 (1438579260584361984)]} 0 1
[junit4:junit4] 2> 1610982 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!9 (1438579260594847744)]} 0 2
[junit4:junit4] 2> 1610984 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[j!9 (1438579260594847744)]} 0 8
[junit4:junit4] 2> 1610986 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!9]} 0 14
[junit4:junit4] 2> 1610992 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10 (1438579260609527808)]} 0 2
[junit4:junit4] 2> 1611019 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!10 (1438579260625256448)]} 0 5
[junit4:junit4] 2> 1611021 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[k!10 (1438579260625256448)]} 0 17
[junit4:junit4] 2> 1611024 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!10]} 0 27
[junit4:junit4] 2> 1611029 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11 (1438579260648325120)]} 0 1
[junit4:junit4] 2> 1611043 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!11 (1438579260658810880)]} 0 1
[junit4:junit4] 2> 1611046 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!11 (1438579260658810880)]} 0 8
[junit4:junit4] 2> 1611048 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!11]} 0 14
[junit4:junit4] 2> 1611054 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12 (1438579260674539520)]} 0 1
[junit4:junit4] 2> 1611076 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!12 (1438579260683976704)]} 0 5
[junit4:junit4] 2> 1611078 T3547 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!12 (1438579260683976704)]} 0 16
[junit4:junit4] 2> 1611085 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!12]} 0 26
[junit4:junit4] 2> 1611091 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13 (1438579260713336832)]} 0 2
[junit4:junit4] 2> 1611104 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!13 (1438579260722774016)]} 0 1
[junit4:junit4] 2> 1611107 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!13 (1438579260722774016)]} 0 8
[junit4:junit4] 2> 1611108 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!13]} 0 13
[junit4:junit4] 2> 1611114 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14 (1438579260737454080)]} 0 1
[junit4:junit4] 2> 1611129 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!14 (1438579260748988416)]} 0 1
[junit4:junit4] 2> 1611131 T3548 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!14 (1438579260748988416)]} 0 8
[junit4:junit4] 2> 1611133 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!14]} 0 13
[junit4:junit4] 2> 1611139 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15 (1438579260763668480)]} 0 1
[junit4:junit4] 2> 1611160 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!15 (1438579260782542848)]} 0 1
[junit4:junit4] 2> 1611161 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!15 (1438579260782542848)]} 0 5
[junit4:junit4] 2> 1611163 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!15]} 0 10
[junit4:junit4] 2> 1611167 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16 (1438579260793028608)]} 0 1
[junit4:junit4] 2> 1611181 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!16 (1438579260803514368)]} 0 1
[junit4:junit4] 2> 1611183 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[q!16 (1438579260803514368)]} 0 7
[junit4:junit4] 2> 1611184 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!16]} 0 12
[junit4:junit4] 2> 1611190 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17 (1438579260817145856)]} 0 1
[junit4:junit4] 2> 1611216 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!17 (1438579260830777344)]} 0 1
[junit4:junit4] 2> 1611218 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[r!17 (1438579260830777344)]} 0 18
[junit4:junit4] 2> 1611219 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!17]} 0 24
[junit4:junit4] 2> 1611225 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438579260853846016)]} 0 1
[junit4:junit4] 2> 1611240 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!18 (1438579260865380352)]} 0 1
[junit4:junit4] 2> 1611242 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!18 (1438579260865380352)]} 0 8
[junit4:junit4] 2> 1611248 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19 (1438579260879011840)]} 0 0
[junit4:junit4] 2> 1611260 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!19 (1438579260887400448)]} 0 1
[junit4:junit4] 2> 1611262 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[t!19 (1438579260887400448)]} 0 6
[junit4:junit4] 2> 1611264 T3550 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!19]} 0 11
[junit4:junit4] 2> 1611271 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20 (1438579260901031936)]} 0 2
[junit4:junit4] 2> 1611289 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!20 (1438579260916760576)]} 0 2
[junit4:junit4] 2> 1611291 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!20 (1438579260916760576)]} 0 8
[junit4:junit4] 2> 1611297 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21 (1438579260929343488)]} 0 1
[junit4:junit4] 2> 1611311 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!21 (1438579260939829248)]} 0 1
[junit4:junit4] 2> 1611312 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!21 (1438579260939829248)]} 0 6
[junit4:junit4] 2> 1611314 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!21]} 0 12
[junit4:junit4] 2> 1611320 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438579260953460736)]} 0 1
[junit4:junit4] 2> 1611328 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!22 (1438579260958703616)]} 0 1
[junit4:junit4] 2> 1611330 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!22 (1438579260958703616)]} 0 6
[junit4:junit4] 2> 1611344 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23 (1438579260975480832)]} 0 4
[junit4:junit4] 2> 1611352 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!23 (1438579260983869440)]} 0 0
[junit4:junit4] 2> 1611354 T3551 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!23 (1438579260983869440)]} 0 6
[junit4:junit4] 2> 1611359 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24 (1438579260994355200)]} 0 1
[junit4:junit4] 2> 1611375 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!24 (1438579261003792384)]} 0 1
[junit4:junit4] 2> 1611377 T3547 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[y!24 (1438579261003792384)]} 0 10
[junit4:junit4] 2> 1611379 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!24]} 0 15
[junit4:junit4] 2> 1611384 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25 (1438579261020569600)]} 0 1
[junit4:junit4] 2> 1611395 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!25 (1438579261028958208)]} 0 0
[junit4:junit4] 2> 1611402 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[z!25 (1438579261028958208)]} 0 11
[junit4:junit4] 2> 1611408 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!25]} 0 20
[junit4:junit4] 2> 1611413 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26 (1438579261050978304)]} 0 2
[junit4:junit4] 2> 1611425 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!26 (1438579261060415488)]} 0 0
[junit4:junit4] 2> 1611427 T3548 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!26 (1438579261060415488)]} 0 6
[junit4:junit4] 2> 1611428 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!26]} 0 10
[junit4:junit4] 2> 1611433 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27 (1438579261071949824)]} 0 1
[junit4:junit4] 2> 1611443 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!27 (1438579261079289856)]} 0 0
[junit4:junit4] 2> 1611445 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!27 (1438579261079289856)]} 0 6
[junit4:junit4] 2> 1611447 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!27]} 0 11
[junit4:junit4] 2> 1611451 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438579261091872768)]} 0 1
[junit4:junit4] 2> 1611465 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!28 (1438579261096067072)]} 0 1
[junit4:junit4] 2> 1611467 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!28 (1438579261096067072)]} 0 12
[junit4:junit4] 2> 1611475 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438579261117038592)]} 0 1
[junit4:junit4] 2> 1611485 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!29 (1438579261122281472)]} 0 1
[junit4:junit4] 2> 1611487 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!29 (1438579261122281472)]} 0 8
[junit4:junit4] 2> 1611491 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30 (1438579261132767232)]} 0 1
[junit4:junit4] 2> 1611500 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!30 (1438579261140107264)]} 0 0
[junit4:junit4] 2> 1611501 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!30 (1438579261140107264)]} 0 4
[junit4:junit4] 2> 1611503 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!30]} 0 9
[junit4:junit4] 2> 1611507 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31 (1438579261150593024)]} 0 1
[junit4:junit4] 2> 1611517 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!31 (1438579261156884480)]} 0 1
[junit4:junit4] 2> 1611521 T3551 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[f!31 (1438579261156884480)]} 0 8
[junit4:junit4] 2> 1611525 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!31]} 0 15
[junit4:junit4] 2> 1611530 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32 (1438579261173661696)]} 0 1
[junit4:junit4] 2> 1611539 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!32 (1438579261179953152)]} 0 1
[junit4:junit4] 2> 1611540 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!32 (1438579261179953152)]} 0 6
[junit4:junit4] 2> 1611549 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33 (1438579261194633216)]} 0 1
[junit4:junit4] 2> 1611557 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!33 (1438579261199876096)]} 0 1
[junit4:junit4] 2> 1611558 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!33 (1438579261199876096)]} 0 5
[junit4:junit4] 2> 1611563 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34 (1438579261208264704)]} 0 1
[junit4:junit4] 2> 1611570 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[i!34 (1438579261213507584)]} 0 0
[junit4:junit4] 2> 1611572 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!34 (1438579261213507584)]} 0 6
[junit4:junit4] 2> 1611577 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438579261223993344)]} 0 0
[junit4:junit4] 2> 1611585 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[j!35 (1438579261229236224)]} 0 1
[junit4:junit4] 2> 1611590 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[j!35 (1438579261229236224)]} 0 9
[junit4:junit4] 2> 1611600 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36 (1438579261248110592)]} 0 1
[junit4:junit4] 2> 1611613 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[k!36 (1438579261253353472)]} 0 1
[junit4:junit4] 2> 1611614 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[k!36 (1438579261253353472)]} 0 9
[junit4:junit4] 2> 1611618 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37 (1438579261265936384)]} 0 1
[junit4:junit4] 2> 1611628 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[l!37 (1438579261274324992)]} 0 1
[junit4:junit4] 2> 1611629 T3547 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[l!37 (1438579261274324992)]} 0 5
[junit4:junit4] 2> 1611631 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[l!37]} 0 9
[junit4:junit4] 2> 1611636 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38 (1438579261284810752)]} 0 1
[junit4:junit4] 2> 1611647 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[m!38 (1438579261293199360)]} 0 0
[junit4:junit4] 2> 1611650 T3548 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[m!38 (1438579261293199360)]} 0 7
[junit4:junit4] 2> 1611655 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[m!38]} 0 15
[junit4:junit4] 2> 1611664 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39 (1438579261314170880)]} 0 1
[junit4:junit4] 2> 1611675 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[n!39 (1438579261323608064)]} 0 0
[junit4:junit4] 2> 1611677 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[n!39 (1438579261323608064)]} 0 5
[junit4:junit4] 2> 1611679 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[n!39]} 0 10
[junit4:junit4] 2> 1611685 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40 (1438579261337239552)]} 0 1
[junit4:junit4] 2> 1611698 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[o!40 (1438579261346676736)]} 0 0
[junit4:junit4] 2> 1611701 T3550 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[o!40 (1438579261346676736)]} 0 7
[junit4:junit4] 2> 1611703 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[o!40]} 0 13
[junit4:junit4] 2> 1611710 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41 (1438579261362405376)]} 0 1
[junit4:junit4] 2> 1611730 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[p!41 (1438579261379182592)]} 0 1
[junit4:junit4] 2> 1611731 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[p!41 (1438579261379182592)]} 0 8
[junit4:junit4] 2> 1611732 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[p!41]} 0 17
[junit4:junit4] 2> 1611738 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42 (1438579261391765504)]} 0 1
[junit4:junit4] 2> 1611752 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[q!42 (1438579261402251264)]} 0 1
[junit4:junit4] 2> 1611753 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[q!42 (1438579261402251264)]} 0 6
[junit4:junit4] 2> 1611757 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43 (1438579261411688448)]} 0 1
[junit4:junit4] 2> 1611763 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[r!43 (1438579261415882752)]} 0 0
[junit4:junit4] 2> 1611765 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[r!43 (1438579261415882752)]} 0 5
[junit4:junit4] 2> 1611769 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44 (1438579261425319936)]} 0 1
[junit4:junit4] 2> 1611779 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[s!44 (1438579261431611392)]} 0 1
[junit4:junit4] 2> 1611781 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[s!44 (1438579261431611392)]} 0 6
[junit4:junit4] 2> 1611784 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[s!44]} 0 11
[junit4:junit4] 2> 1611795 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45 (1438579261451534336)]} 0 1
[junit4:junit4] 2> 1611806 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[t!45 (1438579261458874368)]} 0 1
[junit4:junit4] 2> 1611807 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[t!45 (1438579261458874368)]} 0 6
[junit4:junit4] 2> 1611812 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46 (1438579261470408704)]} 0 1
[junit4:junit4] 2> 1611827 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[u!46 (1438579261480894464)]} 0 1
[junit4:junit4] 2> 1611830 T3551 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[u!46 (1438579261480894464)]} 0 9
[junit4:junit4] 2> 1611832 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[u!46]} 0 14
[junit4:junit4] 2> 1611839 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47 (1438579261497671680)]} 0 1
[junit4:junit4] 2> 1611858 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[v!47 (1438579261508157440)]} 0 5
[junit4:junit4] 2> 1611860 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[v!47 (1438579261508157440)]} 0 12
[junit4:junit4] 2> 1611862 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[v!47]} 0 17
[junit4:junit4] 2> 1611870 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48 (1438579261529128960)]} 0 2
[junit4:junit4] 2> 1611891 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[w!48 (1438579261546954752)]} 0 2
[junit4:junit4] 2> 1611894 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[w!48 (1438579261546954752)]} 0 10
[junit4:junit4] 2> 1611896 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[w!48]} 0 16
[junit4:junit4] 2> 1611902 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49 (1438579261564780544)]} 0 1
[junit4:junit4] 2> 1611915 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[x!49 (1438579261574217728)]} 0 1
[junit4:junit4] 2> 1611917 T3547 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[x!49 (1438579261574217728)]} 0 7
[junit4:junit4] 2> 1611919 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[x!49]} 0 12
[junit4:junit4] 2> 1611925 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438579261587849216)]} 0 2
[junit4:junit4] 2> 1611946 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[y!50 (1438579261600432128)]} 0 1
[junit4:junit4] 2> 1611948 T3550 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[y!50 (1438579261600432128)]} 0 13
[junit4:junit4] 2> 1611953 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51 (1438579261617209344)]} 0 1
[junit4:junit4] 2> 1611965 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[z!51 (1438579261626646528)]} 0 1
[junit4:junit4] 2> 1611967 T3535 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[z!51 (1438579261626646528)]} 0 7
[junit4:junit4] 2> 1611970 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[z!51]} 0 13
[junit4:junit4] 2> 1611976 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52 (1438579261641326592)]} 0 1
[junit4:junit4] 2> 1611987 T3583 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[a!52 (1438579261649715200)]} 0 1
[junit4:junit4] 2> 1611989 T3549 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[a!52 (1438579261649715200)]} 0 7
[junit4:junit4] 2> 1611995 T3530 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[a!52]} 0 15
[junit4:junit4] 2> 1612005 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53 (1438579261672783872)]} 0 1
[junit4:junit4] 2> 1612019 T3562 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[b!53 (1438579261683269632)]} 0 1
[junit4:junit4] 2> 1612021 T3531 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[b!53 (1438579261683269632)]} 0 7
[junit4:junit4] 2> 1612022 T3567 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[b!53]} 0 11
[junit4:junit4] 2> 1612027 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54 (1438579261695852544)]} 0 0
[junit4:junit4] 2> 1612040 T3566 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[c!54 (1438579261704241152)]} 0 1
[junit4:junit4] 2> 1612042 T3532 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[c!54 (1438579261704241152)]} 0 7
[junit4:junit4] 2> 1612043 T3584 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[c!54]} 0 11
[junit4:junit4] 2> 1612048 T3512 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55 (1438579261716824064)]} 0 1
[junit4:junit4] 2> 1612061 T3579 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[d!55 (1438579261722066944)]} 0 2
[junit4:junit4] 2> 1612064 T3546 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[d!55 (1438579261722066944)]} 0 12
[junit4:junit4] 2> 1612070 T3513 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56 (1438579261739892736)]} 0 1
[junit4:junit4] 2> 1612082 T3580 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[e!56 (1438579261749329920)]} 0 1
[junit4:junit4] 2> 1612083 T3551 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50923/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[e!56 (1438579261749329920)]} 0 5
[junit4:junit4] 2> 1612084 T3565 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[e!56]} 0 10
[junit4:junit4] 2> 1612089 T3515 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57 (1438579261760864256)]} 0 1
[junit4:junit4] 2> 1612098 T3581 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50920/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[f!57 (1438579261766107136)]} 0 1
[junit4:junit4] 2> 1612100 T3548 C620 P50920 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[f!57 (1438579261766107136)]} 0 7
[junit4:junit4] 2> 1612104 T3514 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!58 (1438579261776592896)]} 0 1
[junit4:junit4] 2> 1612114 T3564 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[g!58 (1438579261783932928)]} 0 1
[junit4:junit4] 2> 1612116 T3533 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50927/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[g!58 (1438579261783932928)]} 0 5
[junit4:junit4] 2> 1612117 T3582 C619 P50927 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[g!58]} 0 9
[junit4:junit4] 2> 1612126 T3510 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!59 (1438579261798612992)]} 0 2
[junit4:junit4] 2> 1612135 T3563 C623 P50923 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:50917/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[h!59 (1438579261804904448)]} 0 1
[junit4:junit4] 2> 1612137 T3534 C622 P50917 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[h!59 (1438579261804904448)]} 0 6
[junit4:junit4] 2> 1612141 T3511 C621 P50913 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[i!60 (1438579261815
[...truncated too long message...]
/leader_elect/shard1/election/89911225723518988-5-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89911225723518986-4-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51176_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51176"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89911225723518979-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51165_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51165"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89911225723518986-127.0.0.1:51176_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89911225723518990-127.0.0.1:51169_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89911225723518988-127.0.0.1:51183_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89911225723518979-127.0.0.1:51165_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89911225723518979-127.0.0.1:51165_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=ED5D37BBB19952BE -Dtests.slow=true -Dtests.locale=fi_FI -Dtests.timezone=America/Argentina/San_Luis -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 128s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51169/collection1lastClient and got 259 from http://127.0.0.1:51176/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([ED5D37BBB19952BE:6CBBB9A3C6C63282]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 2073776 T4089 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 128110 T4088 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 2074558 T4183 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2074558 T4183 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2074559 T4183 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), text=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=SimpleText), rnd_b=PostingsFormat(name=MockRandom), intDefault=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), timestamp=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), id=PostingsFormat(name=MockRandom), a_t=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), range_facet_sl=PostingsFormat(name=MockRandom), range_facet_si=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), other_tl1=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), multiDefault=Pulsing41(freqCutoff=10 minBlockSize=79 maxBlockSize=254), a_si=PostingsFormat(name=SimpleText)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=DefaultSimilarity, locale=fi_FI, timezone=America/Argentina/San_Luis
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=3,free=173366768,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReload, TestXIncludeConfig, XmlUpdateRequestHandlerTest, TestPropInject, HighlighterConfigTest, ZkControllerTest, ZkCLITest, TestSolrIndexConfig, SuggesterTest, SimpleFacetsTest, NumericFieldsTest, TestAtomicUpdateErrorCases, TestSchemaVersionResource, DirectUpdateHandlerTest, OpenCloseCoreStressTest, RequiredFieldsTest, TimeZoneUtilsTest, TestGroupingSearch, JsonLoaderTest, TestPerFieldSimilarity, TestMergePolicyConfig, PeerSyncTest, SolrCmdDistributorTest, ClusterStateUpdateTest, SolrPluginUtilsTest, SolrCoreCheckLockOnStartupTest, SolrInfoMBeanTest, DisMaxRequestHandlerTest, TestExtendedDismaxParser, ResponseLogComponentTest, TestPostingsSolrHighlighter, OverseerTest, ChaosMonkeySafeLeaderTest, TestFoldingMultitermQuery, OutputWriterTest, TestFastWriter, SolrCoreTest, SearchHandlerTest, TestCollationField, FileBasedSpellCheckerTest, TestSurroundQueryParser, TestManagedSchemaFieldResource, TestDFRSimilarityFactory, PathHierarchyTokenizerFactoryTest, MultiTermTest, TestSolrDeletionPolicy1, TestOmitPositions, TestCoreDiscovery, TestSolrXml, SolrIndexConfigTest, SimplePostToolTest, WordBreakSolrSpellCheckerTest, TestBinaryField, FieldMutatingUpdateProcessorTest, TestRemoteStreaming, TestUtils, TestFastLRUCache, TestManagedSchema, SoftAutoCommitTest, TestDefaultSearchFieldResource, TestCodecSupport, TestPropInjectDefaults, TestSuggestSpellingConverter, TestFieldCollectionResource, TestSchemaResource, DistanceFunctionTest, MBeansHandlerTest, SchemaVersionSpecificBehaviorTest, StandardRequestHandlerTest, CSVRequestHandlerTest, TestFieldResource, DOMUtilTest, TestDynamicFieldCollectionResource, TestTrie, BasicDistributedZkTest, SignatureUpdateProcessorFactoryTest, CircularListTest, TestPartialUpdateDeduplication, TestDistributedSearch, TestCopyFieldCollectionResource, CollectionsAPIDistributedZkTest, TestUniqueKeyFieldResource, TestClassNameShortening, IndexSchemaTest, TestRangeQuery, EchoParamsTest, NotRequiredUniqueKeyTest, TestPHPSerializedResponseWriter, PolyFieldTest, TestSolrQueryParser, DebugComponentTest, TestPseudoReturnFields, TestSolrDeletionPolicy2, BadCopyFieldTest, StatelessScriptUpdateProcessorFactoryTest, FastVectorHighlighterTest, BadComponentTest, TestReversedWildcardFilterFactory, DocValuesMultiTest, SpellingQueryConverterTest, BinaryUpdateRequestHandlerTest, DirectUpdateHandlerOptimizeTest, TestRealTimeGet, TestArbitraryIndexDir, RequestHandlersTest, UUIDFieldTest, TestCloudManagedSchemaAddField, AlternateDirectoryTest, TestSolr4Spatial, TestSchemaSimilarityResource, TermVectorComponentTest, TestFuzzyAnalyzedSuggestions, TestSolrQueryParserDefaultOperatorResource, TestNumberUtils, TestSolrJ, LeaderElectionTest, PingRequestHandlerTest, ResourceLoaderTest, TestFunctionQuery, PrimUtilsTest, SliceStateUpdateTest, SpellCheckComponentTest, PrimitiveFieldTypeTest, PreAnalyzedUpdateProcessorTest, AutoCommitTest, TestValueSourceCache, TestQuerySenderListener, FullSolrCloudDistribCmdsTest, TestQueryUtils, CacheHeaderTest, ShardSplitTest, TestFieldTypeResource, RAMDirectoryFactoryTest, FieldAnalysisRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestRTGBase, TestJoin, TestDocSet, DocumentBuilderTest, TestSchemaNameResource, TestWordDelimiterFilterFactory, TestLuceneMatchVersion, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed in 129.17s, 1 test, 1 failure <<< FAILURES!
[...truncated 456 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 error, 1 failure, 18 ignored (6 assumptions)
Total time: 107 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 586 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/586/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
IOException occured when talking to server at: http://127.0.0.1:50671/_w/i/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:50671/_w/i/collection1
at __randomizedtesting.SeedInfo.seed([1B9D7ED53A28CEB8:9A7BF0CD4D77AE84]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1049)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.SocketException: Invalid argument
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:150)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
... 46 more
Build Log:
[...truncated 17594 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4] 2> 1900977 T4431 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_w/i
[junit4:junit4] 2> 1900989 T4431 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeySafeLeaderTest-1371907356854
[junit4:junit4] 2> 1900991 T4431 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1900992 T4432 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1901093 T4431 oasc.ZkTestServer.run start zk server on port:50615
[junit4:junit4] 2> 1901095 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1901109 T4438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@66032b76 name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1901110 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1901112 T4431 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1901127 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1901138 T4440 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ebe9d9b name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1901138 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1901139 T4431 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1901149 T4431 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1901157 T4431 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1901168 T4431 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1901175 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1901181 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1901193 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1901197 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1901204 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1901205 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1901211 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1901212 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1901219 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1901220 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1901227 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1901228 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1901241 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1901243 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1901250 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1901251 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1901257 T4431 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1901258 T4431 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1901766 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1901777 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50618
[junit4:junit4] 2> 1901779 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1901780 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1901780 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129
[junit4:junit4] 2> 1901781 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/solr.xml
[junit4:junit4] 2> 1901781 T4431 oasc.CoreContainer.<init> New CoreContainer 2014686949
[junit4:junit4] 2> 1901782 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/'
[junit4:junit4] 2> 1901783 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/'
[junit4:junit4] 2> 1901966 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1901967 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1901967 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1901968 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1901969 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1901969 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1901970 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1901970 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1901971 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1901971 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1901991 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1901992 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1901992 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1901995 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1902000 T4451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6695f08f name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1902001 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1902005 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1902029 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1902033 T4453 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1108a17f name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1902033 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1902036 T4431 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1902047 T4431 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1902096 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1902100 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50618__w%2Fi
[junit4:junit4] 2> 1902102 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50618__w%2Fi
[junit4:junit4] 2> 1902110 T4431 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1902131 T4431 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1902138 T4431 oasc.Overseer.start Overseer (id=89909320539570179-127.0.0.1:50618__w%2Fi-n_0000000000) starting
[junit4:junit4] 2> 1902162 T4431 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1902179 T4455 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1902181 T4454 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1902183 T4431 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1902189 T4431 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1902193 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1902206 T4456 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1902207 T4456 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1903714 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1903716 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50618__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50618/_w/i"}
[junit4:junit4] 2> 1903716 T4454 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1903717 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1903730 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1904211 T4456 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/collection1
[junit4:junit4] 2> 1904211 T4456 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1904221 T4456 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1904222 T4456 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1904226 T4456 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/collection1/'
[junit4:junit4] 2> 1904228 T4456 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1904228 T4456 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/collection1/lib/README' to classloader
[junit4:junit4] 2> 1904315 T4456 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1904409 T4456 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1904413 T4456 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1904421 T4456 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1905465 T4456 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1905484 T4456 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1905492 T4456 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1905544 T4456 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1905560 T4456 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1905587 T4456 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1905591 T4456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1905591 T4456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1905592 T4456 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1905596 T4456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1905596 T4456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1905597 T4456 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1905677 T4456 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1371907357129/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data/
[junit4:junit4] 2> 1905678 T4456 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1905678 T4456 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1905680 T4456 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data
[junit4:junit4] 2> 1905681 T4456 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data/index/
[junit4:junit4] 2> 1905681 T4456 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1905682 T4456 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data/index
[junit4:junit4] 2> 1905690 T4456 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/control/data/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1905690 T4456 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1905701 T4456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1905702 T4456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1905703 T4456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1905704 T4456 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1905705 T4456 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1905705 T4456 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1905706 T4456 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1905707 T4456 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1905707 T4456 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1905723 T4456 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1905734 T4456 oass.SolrIndexSearcher.<init> Opening Searcher@5cd08c64 main
[junit4:junit4] 2> 1905737 T4456 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1905737 T4456 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1905744 T4457 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5cd08c64 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1905746 T4456 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1905747 T4456 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50618/_w/i collection:control_collection shard:shard1
[junit4:junit4] 2> 1905748 T4456 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1905765 T4456 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1905771 T4456 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1905771 T4456 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1905773 T4456 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50618/_w/i/collection1/
[junit4:junit4] 2> 1905776 T4456 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1905778 T4456 oasc.SyncStrategy.syncToMe http://127.0.0.1:50618/_w/i/collection1/ has no replicas
[junit4:junit4] 2> 1905778 T4456 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50618/_w/i/collection1/
[junit4:junit4] 2> 1905778 T4456 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1906755 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1906772 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1906833 T4456 oasc.ZkController.register We are http://127.0.0.1:50618/_w/i/collection1/ and leader is http://127.0.0.1:50618/_w/i/collection1/
[junit4:junit4] 2> 1906833 T4456 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50618/_w/i
[junit4:junit4] 2> 1906834 T4456 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1906834 T4456 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1906834 T4456 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1906838 T4456 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1906841 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1906841 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1906842 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1906863 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1906872 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1906880 T4460 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@553e634e name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1906880 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1906883 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1906890 T4431 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1907358 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1907365 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50622
[junit4:junit4] 2> 1907366 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1907367 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1907368 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755
[junit4:junit4] 2> 1907368 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/solr.xml
[junit4:junit4] 2> 1907369 T4431 oasc.CoreContainer.<init> New CoreContainer 1791206176
[junit4:junit4] 2> 1907370 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/'
[junit4:junit4] 2> 1907370 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/'
[junit4:junit4] 2> 1907570 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1907571 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1907571 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1907572 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1907572 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1907573 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1907574 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1907574 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1907575 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1907575 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1907600 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1907602 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1907602 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1907604 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1907609 T4471 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cbbb120 name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1907610 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1907614 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1907632 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1907637 T4473 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2aed7973 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1907638 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1907648 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1908287 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1908289 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50618__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50618/_w/i"}
[junit4:junit4] 2> 1908298 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1908298 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1908304 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1908656 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50622__w%2Fi
[junit4:junit4] 2> 1908659 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50622__w%2Fi
[junit4:junit4] 2> 1908671 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1908673 T4473 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1908673 T4460 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1908674 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1908674 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1908681 T4453 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1908692 T4474 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1908692 T4474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1909813 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1909815 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50622__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50622/_w/i"}
[junit4:junit4] 2> 1909815 T4454 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4] 2> 1909816 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1909824 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1909825 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1909826 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1910698 T4474 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/collection1
[junit4:junit4] 2> 1910699 T4474 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1910700 T4474 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1910701 T4474 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1910704 T4474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/collection1/'
[junit4:junit4] 2> 1910706 T4474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1910707 T4474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/collection1/lib/README' to classloader
[junit4:junit4] 2> 1910800 T4474 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1910896 T4474 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1910899 T4474 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1910912 T4474 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1912106 T4474 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1912121 T4474 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1912127 T4474 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1912160 T4474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1912174 T4474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1912185 T4474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1912189 T4474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1912190 T4474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1912190 T4474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1912193 T4474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1912193 T4474 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1912194 T4474 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1912195 T4474 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1371907362755/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/
[junit4:junit4] 2> 1912195 T4474 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1912197 T4474 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1912198 T4474 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1
[junit4:junit4] 2> 1912200 T4474 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index/
[junit4:junit4] 2> 1912201 T4474 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1912202 T4474 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index
[junit4:junit4] 2> 1912213 T4474 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1912214 T4474 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1912222 T4474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1912222 T4474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1912224 T4474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1912233 T4474 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1912235 T4474 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1912235 T4474 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1912237 T4474 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1912238 T4474 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1912239 T4474 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1912259 T4474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1912270 T4474 oass.SolrIndexSearcher.<init> Opening Searcher@72a90538 main
[junit4:junit4] 2> 1912271 T4474 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1912272 T4474 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1912280 T4475 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72a90538 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1912284 T4474 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1912284 T4474 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50622/_w/i collection:collection1 shard:shard1
[junit4:junit4] 2> 1912286 T4474 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1912315 T4474 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1912322 T4474 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1912323 T4474 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1912323 T4474 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50622/_w/i/collection1/
[junit4:junit4] 2> 1912323 T4474 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1912324 T4474 oasc.SyncStrategy.syncToMe http://127.0.0.1:50622/_w/i/collection1/ has no replicas
[junit4:junit4] 2> 1912324 T4474 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50622/_w/i/collection1/
[junit4:junit4] 2> 1912324 T4474 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1912847 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1912884 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1912885 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1912885 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1912916 T4474 oasc.ZkController.register We are http://127.0.0.1:50622/_w/i/collection1/ and leader is http://127.0.0.1:50622/_w/i/collection1/
[junit4:junit4] 2> 1912921 T4474 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50622/_w/i
[junit4:junit4] 2> 1912924 T4474 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1912925 T4474 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1912926 T4474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1912936 T4474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1912939 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1912941 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1912942 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1913642 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1913651 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50625
[junit4:junit4] 2> 1913655 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1913656 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1913657 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836
[junit4:junit4] 2> 1913658 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/solr.xml
[junit4:junit4] 2> 1913659 T4431 oasc.CoreContainer.<init> New CoreContainer 250206443
[junit4:junit4] 2> 1913660 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/'
[junit4:junit4] 2> 1913661 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/'
[junit4:junit4] 2> 1913925 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1913927 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1913928 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1913928 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1913929 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1913931 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1913932 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1913933 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1913934 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1913935 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1913974 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1913975 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1913976 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1913978 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1913989 T4487 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6441194c name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1913991 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1913999 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1914041 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1914047 T4489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@186e1599 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1914053 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1914071 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1914417 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1914427 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50622__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50622/_w/i"}
[junit4:junit4] 2> 1914442 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1914443 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1914446 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1914447 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1915086 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50625__w%2Fi
[junit4:junit4] 2> 1915090 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50625__w%2Fi
[junit4:junit4] 2> 1915100 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1915107 T4460 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1915108 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1915108 T4489 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1915109 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1915110 T4473 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1915110 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1915124 T4453 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1915144 T4490 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1915147 T4490 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1915968 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1915971 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50625__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50625/_w/i"}
[junit4:junit4] 2> 1915971 T4454 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 1915972 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1916262 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1916263 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1916273 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1916274 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1917158 T4490 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/collection1
[junit4:junit4] 2> 1917159 T4490 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1917162 T4490 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1917162 T4490 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1917165 T4490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/collection1/'
[junit4:junit4] 2> 1917167 T4490 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1917168 T4490 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/collection1/lib/README' to classloader
[junit4:junit4] 2> 1917288 T4490 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1917435 T4490 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1917439 T4490 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1917448 T4490 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1918893 T4490 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1918908 T4490 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1918913 T4490 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1918959 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1918972 T4490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1918990 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1918996 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1918997 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1918998 T4490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1919001 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1919002 T4490 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1919003 T4490 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1919004 T4490 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1371907368836/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/
[junit4:junit4] 2> 1919004 T4490 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1919006 T4490 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1919008 T4490 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2
[junit4:junit4] 2> 1919009 T4490 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index/
[junit4:junit4] 2> 1919014 T4490 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1919019 T4490 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index
[junit4:junit4] 2> 1919045 T4490 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1919047 T4490 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1919055 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1919056 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1919059 T4490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1919138 T4490 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1919140 T4490 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1919141 T4490 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1919143 T4490 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1919145 T4490 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1919146 T4490 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1919169 T4490 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1919187 T4490 oass.SolrIndexSearcher.<init> Opening Searcher@6c0772e2 main
[junit4:junit4] 2> 1919190 T4490 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1919191 T4490 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1919207 T4491 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c0772e2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1919217 T4490 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1919218 T4490 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50625/_w/i collection:collection1 shard:shard2
[junit4:junit4] 2> 1919220 T4490 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 1919239 T4490 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1919246 T4490 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1919246 T4490 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1919247 T4490 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50625/_w/i/collection1/
[junit4:junit4] 2> 1919247 T4490 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1919248 T4490 oasc.SyncStrategy.syncToMe http://127.0.0.1:50625/_w/i/collection1/ has no replicas
[junit4:junit4] 2> 1919248 T4490 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50625/_w/i/collection1/
[junit4:junit4] 2> 1919248 T4490 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 1919297 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1919322 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1919323 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1919324 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1919325 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1919337 T4490 oasc.ZkController.register We are http://127.0.0.1:50625/_w/i/collection1/ and leader is http://127.0.0.1:50625/_w/i/collection1/
[junit4:junit4] 2> 1919338 T4490 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50625/_w/i
[junit4:junit4] 2> 1919338 T4490 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1919339 T4490 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1919339 T4490 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1919351 T4490 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1919353 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1919354 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1919355 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1920015 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1920023 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50628
[junit4:junit4] 2> 1920026 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1920027 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1920027 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250
[junit4:junit4] 2> 1920028 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/solr.xml
[junit4:junit4] 2> 1920028 T4431 oasc.CoreContainer.<init> New CoreContainer 265960198
[junit4:junit4] 2> 1920029 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/'
[junit4:junit4] 2> 1920030 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/'
[junit4:junit4] 2> 1920277 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1920278 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1920279 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1920279 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1920280 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1920281 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1920281 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1920282 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1920283 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1920283 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1920313 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1920315 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1920316 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1920319 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1920339 T4503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e9f5620 name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1920340 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1920346 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1920373 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1920381 T4505 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e740978 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1920381 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1920405 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1920851 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1920856 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50625__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50625/_w/i"}
[junit4:junit4] 2> 1920878 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1920878 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1920878 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1920879 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1920879 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1921419 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50628__w%2Fi
[junit4:junit4] 2> 1921423 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50628__w%2Fi
[junit4:junit4] 2> 1921452 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1921468 T4505 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1921469 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1921469 T4460 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1921470 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1921470 T4489 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1921471 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1921471 T4473 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1921472 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1921475 T4453 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1921506 T4506 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1921507 T4506 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1922399 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1922401 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50628__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50628/_w/i"}
[junit4:junit4] 2> 1922402 T4454 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 1922402 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 1922415 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1922421 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1922422 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1922423 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1922424 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1922511 T4506 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/collection1
[junit4:junit4] 2> 1922512 T4506 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1922515 T4506 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1922515 T4506 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1922519 T4506 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/collection1/'
[junit4:junit4] 2> 1922522 T4506 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1922523 T4506 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/collection1/lib/README' to classloader
[junit4:junit4] 2> 1922709 T4506 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1922867 T4506 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1922871 T4506 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1922884 T4506 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1924472 T4506 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1924496 T4506 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1924506 T4506 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1924565 T4506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1924598 T4506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1924613 T4506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1924617 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1924618 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1924619 T4506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1924623 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1924624 T4506 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1924624 T4506 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1924625 T4506 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1371907375250/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3/
[junit4:junit4] 2> 1924626 T4506 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1924627 T4506 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1924630 T4506 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3
[junit4:junit4] 2> 1924631 T4506 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3/index/
[junit4:junit4] 2> 1924634 T4506 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1924636 T4506 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3/index
[junit4:junit4] 2> 1924647 T4506 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty3/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1924648 T4506 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1924655 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1924656 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1924658 T4506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1924660 T4506 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1924666 T4506 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1924669 T4506 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1924672 T4506 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1924674 T4506 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1924675 T4506 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1924699 T4506 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1924717 T4506 oass.SolrIndexSearcher.<init> Opening Searcher@24d69f96 main
[junit4:junit4] 2> 1924720 T4506 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1924721 T4506 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1924732 T4507 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24d69f96 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1924739 T4506 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1924739 T4506 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50628/_w/i collection:collection1 shard:shard3
[junit4:junit4] 2> 1924745 T4506 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4] 2> 1924777 T4506 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1924786 T4506 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1924787 T4506 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1924787 T4506 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50628/_w/i/collection1/
[junit4:junit4] 2> 1924788 T4506 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1924789 T4506 oasc.SyncStrategy.syncToMe http://127.0.0.1:50628/_w/i/collection1/ has no replicas
[junit4:junit4] 2> 1924789 T4506 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50628/_w/i/collection1/
[junit4:junit4] 2> 1924790 T4506 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4] 2> 1925451 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1925476 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1925477 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1925477 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1925478 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1925479 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1925532 T4506 oasc.ZkController.register We are http://127.0.0.1:50628/_w/i/collection1/ and leader is http://127.0.0.1:50628/_w/i/collection1/
[junit4:junit4] 2> 1925533 T4506 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50628/_w/i
[junit4:junit4] 2> 1925533 T4506 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1925534 T4506 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1925534 T4506 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1925540 T4506 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1925544 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1925545 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1925546 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1926193 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1926202 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50631
[junit4:junit4] 2> 1926206 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1926207 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1926208 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440
[junit4:junit4] 2> 1926208 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/solr.xml
[junit4:junit4] 2> 1926209 T4431 oasc.CoreContainer.<init> New CoreContainer 1341166329
[junit4:junit4] 2> 1926212 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/'
[junit4:junit4] 2> 1926213 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/'
[junit4:junit4] 2> 1926467 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1926468 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1926468 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1926468 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1926469 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1926469 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1926470 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1926470 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1926471 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1926471 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1926497 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1926503 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1926506 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1926509 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1926519 T4519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@390e6f name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1926520 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1926543 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1926566 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1926575 T4521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56ba1c24 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1926578 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1926595 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1927005 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1927009 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50628__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50628/_w/i"}
[junit4:junit4] 2> 1927022 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927023 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927023 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927024 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927024 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927024 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927605 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50631__w%2Fi
[junit4:junit4] 2> 1927608 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50631__w%2Fi
[junit4:junit4] 2> 1927615 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1927620 T4460 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927621 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1927621 T4473 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927621 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1927631 T4453 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927645 T4521 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927646 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1927646 T4489 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927647 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1927645 T4505 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1927648 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1927660 T4522 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1927660 T4522 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1928550 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1928553 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50631__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50631/_w/i"}
[junit4:junit4] 2> 1928555 T4454 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 1928556 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 1928569 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928570 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928571 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928571 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928572 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928573 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1928670 T4522 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/collection1
[junit4:junit4] 2> 1928670 T4522 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1928672 T4522 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1928672 T4522 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1928677 T4522 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/collection1/'
[junit4:junit4] 2> 1928679 T4522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1928679 T4522 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/collection1/lib/README' to classloader
[junit4:junit4] 2> 1928799 T4522 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1928930 T4522 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1928934 T4522 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1928945 T4522 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1930564 T4522 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1930595 T4522 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1930606 T4522 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1930654 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1930666 T4522 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1930678 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1930683 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1930683 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1930684 T4522 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1930689 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1930690 T4522 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1930690 T4522 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1930691 T4522 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1371907381440/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/
[junit4:junit4] 2> 1930692 T4522 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1930693 T4522 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1930695 T4522 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4
[junit4:junit4] 2> 1930697 T4522 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index/
[junit4:junit4] 2> 1930700 T4522 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1930701 T4522 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index
[junit4:junit4] 2> 1930717 T4522 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1930718 T4522 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1930726 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1930727 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1930729 T4522 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1930732 T4522 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1930733 T4522 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1930734 T4522 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1930735 T4522 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1930736 T4522 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1930737 T4522 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1930752 T4522 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1930763 T4522 oass.SolrIndexSearcher.<init> Opening Searcher@2d22fd6f main
[junit4:junit4] 2> 1930765 T4522 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1930765 T4522 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1930774 T4523 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d22fd6f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1930779 T4522 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1930780 T4522 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50631/_w/i collection:collection1 shard:shard1
[junit4:junit4] 2> 1930787 T4522 oasc.ZkController.register We are http://127.0.0.1:50631/_w/i/collection1/ and leader is http://127.0.0.1:50622/_w/i/collection1/
[junit4:junit4] 2> 1930787 T4522 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50631/_w/i
[junit4:junit4] 2> 1930788 T4522 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1930788 T4522 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C929 name=collection1 org.apache.solr.core.SolrCore@49cb979a url=http://127.0.0.1:50631/_w/i/collection1 node=127.0.0.1:50631__w%2Fi C929_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50631__w%2Fi, base_url=http://127.0.0.1:50631/_w/i}
[junit4:junit4] 2> 1930790 T4524 C929 P50631 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1930790 T4522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1930790 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1930791 T4524 C929 P50631 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1930792 T4524 C929 P50631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1930793 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1930794 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1930795 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1930817 T4524 C929 P50631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1930855 T4468 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1931514 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1931540 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50635
[junit4:junit4] 2> 1931544 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1931547 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1931548 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679
[junit4:junit4] 2> 1931549 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/solr.xml
[junit4:junit4] 2> 1931550 T4431 oasc.CoreContainer.<init> New CoreContainer 326200447
[junit4:junit4] 2> 1931552 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/'
[junit4:junit4] 2> 1931555 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/'
[junit4:junit4] 2> 1931624 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1931627 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50631__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50631/_w/i"}
[junit4:junit4] 2> 1931645 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931646 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931647 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931648 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931649 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931646 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1931857 T4468 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1931858 T4468 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&nodeName=127.0.0.1:50631__w%252Fi&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 1931875 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1931879 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1931882 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1931883 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1931884 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1931885 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1931886 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1931887 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1931888 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1931889 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1931926 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1931928 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1931929 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1931932 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1931940 T4536 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e1f40bb name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1931941 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1931955 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1931982 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1931989 T4538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5100ed31 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1931989 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1932003 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1933019 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50635__w%2Fi
[junit4:junit4] 2> 1933022 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50635__w%2Fi
[junit4:junit4] 2> 1933037 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1933041 T4505 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933041 T4538 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933042 T4460 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933042 T4473 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933041 T4489 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933043 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933041 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933043 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933043 T4521 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933046 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933043 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933053 T4453 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4] 2> 1933062 T4539 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1933063 T4539 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1933161 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1933165 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50635__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50635/_w/i"}
[junit4:junit4] 2> 1933166 T4454 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 1933166 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 1933180 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933181 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933182 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933181 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933181 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933180 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1933180 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> C929_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50631__w%2Fi, base_url=http://127.0.0.1:50631/_w/i}
[junit4:junit4] 2> 1933861 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50622/_w/i/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1933862 T4524 C929 P50631 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50631/_w/i START replicas=[http://127.0.0.1:50622/_w/i/collection1/] nUpdates=100
[junit4:junit4] 2> 1933863 T4524 C929 P50631 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1933863 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1933864 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1933864 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1933864 T4524 C929 P50631 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1933865 T4524 C929 P50631 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50622/_w/i/collection1/. core=collection1
[junit4:junit4] 2> 1933865 T4524 C929 P50631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C930 name=collection1 org.apache.solr.core.SolrCore@42871554 url=http://127.0.0.1:50622/_w/i/collection1 node=127.0.0.1:50622__w%2Fi C930_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50622__w%2Fi, base_url=http://127.0.0.1:50622/_w/i, leader=true}
[junit4:junit4] 2> 1933900 T4465 C930 P50622 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1933913 T4466 C930 P50622 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1933922 T4466 C930 P50622 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1933923 T4466 C930 P50622 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1933927 T4466 C930 P50622 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty1/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 1933928 T4466 C930 P50622 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1933930 T4466 C930 P50622 oass.SolrIndexSearcher.<init> Opening Searcher@7e6727dd realtime
[junit4:junit4] 2> 1933931 T4466 C930 P50622 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1933931 T4466 C930 P50622 oasup.LogUpdateProcessor.finish [collection1] webapp=/_w/i path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1933933 T4524 C929 P50631 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1933934 T4524 C929 P50631 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1933940 T4463 C930 P50622 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1933941 T4463 C930 P50622 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1933942 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1933942 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1933943 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1933948 T4464 C930 P50622 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2
[junit4:junit4] 2> 1933950 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1933954 T4524 C929 P50631 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815
[junit4:junit4] 2> 1933955 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4740cf7a fullCopy=false
[junit4:junit4] 2> 1933962 T4468 C930 P50622 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1933967 T4524 C929 P50631 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1933972 T4524 C929 P50631 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1933973 T4524 C929 P50631 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1933981 T4524 C929 P50631 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 1933982 T4524 C929 P50631 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1933983 T4524 C929 P50631 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1933988 T4524 C929 P50631 oass.SolrIndexSearcher.<init> Opening Searcher@4789ad23 main
[junit4:junit4] 2> 1933991 T4523 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4789ad23 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1933992 T4524 C929 P50631 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815;done=true>>]
[junit4:junit4] 2> 1933993 T4524 C929 P50631 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815
[junit4:junit4] 2> 1933993 T4524 C929 P50631 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty4/index.20130622092309815
[junit4:junit4] 2> 1933995 T4524 C929 P50631 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1933996 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1933996 T4524 C929 P50631 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1933996 T4524 C929 P50631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1934000 T4524 C929 P50631 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1934067 T4539 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/collection1
[junit4:junit4] 2> 1934068 T4539 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1934071 T4539 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1934071 T4539 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1934076 T4539 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/collection1/'
[junit4:junit4] 2> 1934078 T4539 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1934079 T4539 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/collection1/lib/README' to classloader
[junit4:junit4] 2> 1934198 T4539 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1934291 T4539 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1934294 T4539 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1934301 T4539 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1934775 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1934778 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50631__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50631/_w/i"}
[junit4:junit4] 2> 1934787 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934788 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934788 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934788 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934789 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934791 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1934791 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1935588 T4539 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1935612 T4539 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1935617 T4539 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1935656 T4539 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1935670 T4539 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1935678 T4539 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1935681 T4539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1935681 T4539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1935682 T4539 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1935684 T4539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1935685 T4539 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1935685 T4539 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1935686 T4539 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1371907386679/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/
[junit4:junit4] 2> 1935686 T4539 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1935687 T4539 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1935688 T4539 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5
[junit4:junit4] 2> 1935689 T4539 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index/
[junit4:junit4] 2> 1935690 T4539 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1935691 T4539 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index
[junit4:junit4] 2> 1935698 T4539 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1935698 T4539 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1935703 T4539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1935704 T4539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1935706 T4539 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1935708 T4539 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1935711 T4539 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1935711 T4539 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1935717 T4539 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1935720 T4539 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1935721 T4539 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1935744 T4539 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1935785 T4539 oass.SolrIndexSearcher.<init> Opening Searcher@6120ae91 main
[junit4:junit4] 2> 1935790 T4539 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1935791 T4539 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1935810 T4542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6120ae91 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1935815 T4539 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1935815 T4539 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50635/_w/i collection:collection1 shard:shard2
[junit4:junit4] 2> 1935840 T4539 oasc.ZkController.register We are http://127.0.0.1:50635/_w/i/collection1/ and leader is http://127.0.0.1:50625/_w/i/collection1/
[junit4:junit4] 2> 1935840 T4539 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50635/_w/i
[junit4:junit4] 2> 1935841 T4539 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1935841 T4539 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C931 name=collection1 org.apache.solr.core.SolrCore@78feb9d9 url=http://127.0.0.1:50635/_w/i/collection1 node=127.0.0.1:50635__w%2Fi C931_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50635__w%2Fi, base_url=http://127.0.0.1:50635/_w/i}
[junit4:junit4] 2> 1935842 T4543 C931 P50635 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1935843 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1935844 T4543 C931 P50635 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1935844 T4543 C931 P50635 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1935846 T4539 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1935847 T4543 C931 P50635 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1935870 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1935871 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1935872 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1935892 T4481 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1936323 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1936326 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50635__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50635/_w/i"}
[junit4:junit4] 2> 1936345 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936347 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936347 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936349 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936350 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936351 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936352 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1936548 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1936562 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50642
[junit4:junit4] 2> 1936564 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1936564 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1936565 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758
[junit4:junit4] 2> 1936566 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/solr.xml
[junit4:junit4] 2> 1936566 T4431 oasc.CoreContainer.<init> New CoreContainer 270391433
[junit4:junit4] 2> 1936567 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/'
[junit4:junit4] 2> 1936567 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/'
[junit4:junit4] 2> 1936844 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1936846 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1936847 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1936848 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1936849 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1936850 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1936851 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1936852 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1936854 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1936855 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1936893 T4481 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 6, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1936894 T4481 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=6&state=recovering&nodeName=127.0.0.1:50635__w%252Fi&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 1936897 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1936898 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1936901 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1936905 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1936912 T4555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ffe7416 name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1936918 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1936942 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1936973 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1936976 T4557 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7899615b name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1936977 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1936987 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1938003 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50642__w%2Fi
[junit4:junit4] 2> 1938007 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50642__w%2Fi
[junit4:junit4] 2> 1938016 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4] 2> 1938029 T4557 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938030 T4473 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938030 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938031 T4489 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938031 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938031 T4460 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938032 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938033 T4505 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938033 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938034 T4521 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938034 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938050 T4453 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938051 T4538 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4] 2> 1938051 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1938075 T4558 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1938076 T4558 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C931_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50635__w%2Fi, base_url=http://127.0.0.1:50635/_w/i}
[junit4:junit4] 2> 1938898 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50625/_w/i/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1938901 T4543 C931 P50635 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50635/_w/i START replicas=[http://127.0.0.1:50625/_w/i/collection1/] nUpdates=100
[junit4:junit4] 2> 1938903 T4543 C931 P50635 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1938903 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1938904 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1938904 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1938904 T4543 C931 P50635 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1938904 T4543 C931 P50635 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50625/_w/i/collection1/. core=collection1
[junit4:junit4] 2> 1938905 T4543 C931 P50635 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C932 name=collection1 org.apache.solr.core.SolrCore@3541ad7f url=http://127.0.0.1:50625/_w/i/collection1 node=127.0.0.1:50625__w%2Fi C932_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50625__w%2Fi, base_url=http://127.0.0.1:50625/_w/i, leader=true}
[junit4:junit4] 2> 1938938 T4484 C932 P50625 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1938947 T4485 C932 P50625 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1938957 T4485 C932 P50625 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1938959 T4485 C932 P50625 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1938969 T4485 C932 P50625 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty2/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 1938972 T4485 C932 P50625 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1938974 T4485 C932 P50625 oass.SolrIndexSearcher.<init> Opening Searcher@3bc269c5 realtime
[junit4:junit4] 2> 1938975 T4485 C932 P50625 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1938976 T4485 C932 P50625 oasup.LogUpdateProcessor.finish [collection1] webapp=/_w/i path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 29
[junit4:junit4] 2> 1938980 T4543 C931 P50635 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1938986 T4543 C931 P50635 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1938994 T4482 C932 P50625 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1938995 T4482 C932 P50625 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 1938997 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1938998 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1938998 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1939011 T4483 C932 P50625 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 1939013 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1939016 T4543 C931 P50635 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877
[junit4:junit4] 2> 1939017 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Starting download to org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877 lockFactory=org.apache.lucene.store.NativeFSLockFactory@14676c02 fullCopy=false
[junit4:junit4] 2> 1939022 T4481 C932 P50625 oasc.SolrCore.execute [collection1] webapp=/_w/i path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1939026 T4543 C931 P50635 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1939029 T4543 C931 P50635 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1939029 T4543 C931 P50635 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 1939036 T4543 C931 P50635 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index,segFN=segments_2,generation=2}
[junit4:junit4] 2> 1939038 T4543 C931 P50635 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1939039 T4543 C931 P50635 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1939040 T4543 C931 P50635 oass.SolrIndexSearcher.<init> Opening Searcher@9fb253d main
[junit4:junit4] 2> 1939041 T4542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9fb253d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1939042 T4543 C931 P50635 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877;done=true>>]
[junit4:junit4] 2> 1939042 T4543 C931 P50635 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877
[junit4:junit4] 2> 1939043 T4543 C931 P50635 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty5/index.20130622092314877
[junit4:junit4] 2> 1939044 T4543 C931 P50635 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1939044 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1939045 T4543 C931 P50635 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1939045 T4543 C931 P50635 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1939048 T4543 C931 P50635 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1939389 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1939392 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"7",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50642__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50642/_w/i"}
[junit4:junit4] 2> 1939393 T4454 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4] 2> 1939393 T4454 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard3
[junit4:junit4] 2> 1939410 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"6",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50635__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50635/_w/i"}
[junit4:junit4] 2> 1939428 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939429 T4557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939429 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939429 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939430 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939430 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939429 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1939429 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1940082 T4558 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/collection1
[junit4:junit4] 2> 1940082 T4558 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1940085 T4558 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1940085 T4558 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1940097 T4558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/collection1/'
[junit4:junit4] 2> 1940100 T4558 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1940104 T4558 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/collection1/lib/README' to classloader
[junit4:junit4] 2> 1940423 T4558 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1940595 T4558 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1940599 T4558 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1940614 T4558 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1942114 T4558 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1942135 T4558 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1942141 T4558 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1942192 T4558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1942200 T4558 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1942209 T4558 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1942212 T4558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1942213 T4558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1942213 T4558 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1942218 T4558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1942218 T4558 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1942219 T4558 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1942221 T4558 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1371907391758/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6/
[junit4:junit4] 2> 1942221 T4558 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@19582381
[junit4:junit4] 2> 1942223 T4558 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4] 2> 1942224 T4558 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6
[junit4:junit4] 2> 1942226 T4558 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6/index/
[junit4:junit4] 2> 1942227 T4558 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1942228 T4558 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6/index
[junit4:junit4] 2> 1942242 T4558 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1371907356853/jetty6/index,segFN=segments_1,generation=1}
[junit4:junit4] 2> 1942243 T4558 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1942247 T4558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1942248 T4558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1942250 T4558 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1942251 T4558 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1942254 T4558 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1942255 T4558 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1942263 T4558 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1942265 T4558 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1942266 T4558 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1942295 T4558 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1942313 T4558 oass.SolrIndexSearcher.<init> Opening Searcher@64e3c633 main
[junit4:junit4] 2> 1942315 T4558 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1942315 T4558 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1942322 T4561 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64e3c633 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1942329 T4558 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1942330 T4558 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50642/_w/i collection:collection1 shard:shard3
[junit4:junit4] 2> 1942338 T4558 oasc.ZkController.register We are http://127.0.0.1:50642/_w/i/collection1/ and leader is http://127.0.0.1:50628/_w/i/collection1/
[junit4:junit4] 2> 1942338 T4558 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50642/_w/i
[junit4:junit4] 2> 1942339 T4558 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1942339 T4558 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C933 name=collection1 org.apache.solr.core.SolrCore@5ae513c1 url=http://127.0.0.1:50642/_w/i/collection1 node=127.0.0.1:50642__w%2Fi C933_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50642__w%2Fi, base_url=http://127.0.0.1:50642/_w/i}
[junit4:junit4] 2> 1942340 T4562 C933 P50642 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1942341 T4562 C933 P50642 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1942342 T4562 C933 P50642 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1942342 T4562 C933 P50642 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1942344 T4558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1942346 T4562 C933 P50642 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1942349 T4431 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1942351 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1942352 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1942374 T4496 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 7, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1942478 T4454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1942480 T4454 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"7",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50642__w%2Fi",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50642/_w/i"}
[junit4:junit4] 2> 1942510 T4557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942511 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942511 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942511 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942512 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942512 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942513 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1942513 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1943095 T4431 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1943103 T4431 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50649
[junit4:junit4] 2> 1943106 T4431 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1943107 T4431 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1943108 T4431 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371907398235
[junit4:junit4] 2> 1943109 T4431 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371907398235/solr.xml
[junit4:junit4] 2> 1943110 T4431 oasc.CoreContainer.<init> New CoreContainer 342839032
[junit4:junit4] 2> 1943112 T4431 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371907398235/'
[junit4:junit4] 2> 1943119 T4431 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty7-1371907398235/'
[junit4:junit4] 2> 1943309 T4431 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1943310 T4431 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1943310 T4431 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1943311 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1943311 T4431 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1943311 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1943312 T4431 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1943312 T4431 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1943312 T4431 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1943313 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1943338 T4431 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1943340 T4431 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50615/solr
[junit4:junit4] 2> 1943341 T4431 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1943347 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1943360 T4574 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18509953 name:ZooKeeperConnection Watcher:127.0.0.1:50615 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1943361 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1943372 T4431 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1943376 T4496 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 7, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1943377 T4496 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=7&state=recovering&nodeName=127.0.0.1:50642__w%252Fi&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 1943402 T4431 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1943407 T4576 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@662c8ed6 name:ZooKeeperConnection Watcher:127.0.0.1:50615/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1943408 T4431 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1943439 T4431 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1944453 T4431 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50649__w%2Fi
[junit4:junit4] 2> 1944455 T4431 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50649__w%2Fi
[junit4:junit4] 2> 1944465 T4576 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944465 T4505 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944466 T4505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944467 T4557 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1944467 T4453 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4] 2> 1944482 T4538 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944483 T4538 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944483 T4473 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944484 T4473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944485 T4460 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944485 T4460 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944486 T4521 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944486 T4521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944483 T4489 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944488 T4489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 8)
[junit4:junit4] 2> 1944505 T4557 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944505 T4453 oascc.ZkStateReader$3.process Updating live nodes... (8)
[junit4:junit4] 2> 1944521 T4577 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1944522 T4577 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> C933_STATE=coll:collection1 core:collection1 props:{shard=shard3, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50642__w%2Fi, base_url=http://127.0.0.1:50642/_w/i}
[junit4:junit4] 2> 1945380 T4562 C933 P50642 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50628/_w/i/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1945381 T4562 C933 P50642 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50642/_w/i START replicas=[http://127.0.0.1:50628/_w/i/collection1/] nUpdates=100
[junit4:junit4] 2> 1945382 T4562 C933 P50642 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1945383 T4562 C933 P50642 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1945383 T4562 C933 P50642 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1945383 T4562 C933 P50642 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1945384 T4562 C933 P50642 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1945384 T4562 C933 P50642 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50628/_w/i/collection1/. core=collection1
[junit4:junit4] 2> 1945385 T4562 C933 P50642 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C934 name=collection1 org.apache.solr.core.SolrCore@aaf9b27 url=http://127.0.0.1:50628/_w/i/collect
[...truncated too long message...]
ERROR 229s | ChaosMonkeySafeLeaderTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:50671/_w/i/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([1B9D7ED53A28CEB8:9A7BF0CD4D77AE84]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:435)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1049)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:137)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] > Caused by: java.net.SocketException: Invalid argument
[junit4:junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4] > at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
[junit4:junit4] > at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
[junit4:junit4] > at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
[junit4:junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
[junit4:junit4] > at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
[junit4:junit4] > at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
[junit4:junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
[junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
[junit4:junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
[junit4:junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] > at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:365)
[junit4:junit4] > ... 46 more
[junit4:junit4] 2> 2129521 T4431 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 228553 T4430 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 2129584 T4672 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2129585 T4672 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2129586 T4672 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=MockFixedIntBlock(blockSize=829), text=PostingsFormat(name=Direct), _version_=Pulsing41(freqCutoff=3 minBlockSize=74 maxBlockSize=162), rnd_b=Lucene41(blocksize=128), intDefault=Pulsing41(freqCutoff=3 minBlockSize=74 maxBlockSize=162), timestamp=MockFixedIntBlock(blockSize=829), id=Lucene41(blocksize=128), a_t=MockFixedIntBlock(blockSize=829), range_facet_sl=Lucene41(blocksize=128), range_facet_si=MockFixedIntBlock(blockSize=829), other_tl1=Pulsing41(freqCutoff=3 minBlockSize=74 maxBlockSize=162), multiDefault=MockFixedIntBlock(blockSize=829), a_si=Pulsing41(freqCutoff=3 minBlockSize=74 maxBlockSize=162)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=DefaultSimilarity, locale=de_LU, timezone=EST5EDT
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=2,free=325851016,total=536870912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SolrCoreTest, DistanceFunctionTest, TestNumberUtils, SliceStateTest, HardAutoCommitTest, MultiTermTest, SampleTest, TestPseudoReturnFields, SpellingQueryConverterTest, SolrIndexConfigTest, TestBinaryField, TestSolrQueryParser, TestRemoteStreaming, FieldAnalysisRequestHandlerTest, OverseerTest, TestSchemaSimilarityResource, CacheHeaderTest, FastVectorHighlighterTest, OutputWriterTest, PathHierarchyTokenizerFactoryTest, SearchHandlerTest, TestSolrCoreProperties, TestRealTimeGet, CollectionsAPIDistributedZkTest, NoCacheHeaderTest, StandardRequestHandlerTest, TestStressReorder, SolrRequestParserTest, DocumentBuilderTest, SignatureUpdateProcessorFactoryTest, UpdateParamsTest, CopyFieldTest, TestSurroundQueryParser, TestCSVResponseWriter, TestSerializedLuceneMatchVersion, TestSolrQueryParserDefaultOperatorResource, TestSolrJ, TestIndexSearcher, TestFastLRUCache, TestPropInjectDefaults, UnloadDistributedZkTest, RequestHandlersTest, TimeZoneUtilsTest, QueryParsingTest, DisMaxRequestHandlerTest, TestSolrDeletionPolicy1, TestBM25SimilarityFactory, TestSchemaVersionResource, DocValuesMultiTest, ReturnFieldsTest, SuggesterFSTTest, DirectSolrSpellCheckerTest, ExternalFileFieldSortTest, TestMaxScoreQueryParser, TestPerFieldSimilarity, RequiredFieldsTest, QueryElevationComponentTest, TestFieldResource, TestPHPSerializedResponseWriter, SolrInfoMBeanTest, TestDocumentBuilder, TestSolrQueryParserResource, TestBadConfig, ClusterStateTest, TestRTGBase, UniqFieldsUpdateProcessorFactoryTest, TestValueSourceCache, TestLMJelinekMercerSimilarityFactory, SolrIndexSplitterTest, AliasIntegrationTest, DocValuesTest, NumericFieldsTest, DistributedQueryElevationComponentTest, ShowFileRequestHandlerTest, TestSuggestSpellingConverter, TestReplicationHandler, SyncSliceTest, BadIndexSchemaTest, BasicFunctionalityTest, DefaultValueUpdateProcessorTest, TestHashPartitioner, TestLFUCache, TestMergePolicyConfig, TestJmxMonitoredMap, TestSolrDeletionPolicy2, TestSweetSpotSimilarityFactory, TestExtendedDismaxParser, TermVectorComponentDistributedTest, RegexBoostProcessorTest, SuggesterWFSTTest, TestQuerySenderNoQuery, TestQueryUtils, ShardRoutingTest, ZkControllerTest, TestDynamicFieldResource, UpdateRequestProcessorFactoryTest, SliceStateUpdateTest, TestCSVLoader, TestFuzzyAnalyzedSuggestions, TestFunctionQuery, TestQuerySenderListener, XmlUpdateRequestHandlerTest, TestReversedWildcardFilterFactory, SolrPluginUtilsTest, DOMUtilTest, BasicDistributedZkTest, TestQueryTypes, TestFieldCollectionResource, TestSolr4Spatial, TestManagedSchema, SpellCheckCollatorTest, PrimUtilsTest, SpellCheckComponentTest, FieldMutatingUpdateProcessorTest, TestSolrXml, LeaderElectionIntegrationTest, TestFastWriter, TestCharFilters, TestCloudManagedSchemaAddField, TestConfig, TestStressVersions, XsltUpdateRequestHandlerTest, TestDistributedSearch, ClusterStateUpdateTest, TestStressRecovery, WordBreakSolrSpellCheckerTest, OverseerCollectionProcessorTest, TestAnalyzedSuggestions, OpenExchangeRatesOrgProviderTest, IndexSchemaRuntimeFieldTest, QueryEqualityTest, SystemInfoHandlerTest, TestSchemaNameResource, EchoParamsTest, TestLuceneMatchVersion, TestFaceting, TestUniqueKeyFieldResource, IndexReaderFactoryTest, TestPluginEnable, TestManagedSchemaFieldResource, LoggingHandlerTest, SuggesterTSTTest, SuggesterTest, TestDFRSimilarityFactory, CSVRequestHandlerTest, ChaosMonkeySafeLeaderTest]
[junit4:junit4] Completed in 228.86s, 1 test, 1 error <<< FAILURES!
[...truncated 468 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1253 tests, 1 error, 18 ignored (6 assumptions)
Total time: 107 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 585 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/585/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
collection already exists: awholynewcollection_1
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: awholynewcollection_1
at __randomizedtesting.SeedInfo.seed([D4559FF12C31427A:55B311E95B6E2246]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:264)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:313)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1455)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:438)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9573 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 2> 2493372 T4654 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /fd_/
[junit4:junit4] 2> 2493412 T4654 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1371883573866
[junit4:junit4] 2> 2493416 T4654 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 2493417 T4655 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 2493518 T4654 oasc.ZkTestServer.run start zk server on port:50508
[junit4:junit4] 2> 2493521 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2493535 T4661 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f03ef72 name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2493539 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2493539 T4654 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 2493556 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2493560 T4663 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c7a68b9 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2493560 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2493561 T4654 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 2493569 T4654 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 2493577 T4654 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 2493584 T4654 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 2493598 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2493600 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2493615 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 2493616 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 2493625 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2493626 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2493635 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 2493636 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 2493644 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 2493645 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 2493658 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2493662 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2493674 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2493675 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2493685 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2493686 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2493694 T4654 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2493695 T4654 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2494323 T4654 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2494346 T4654 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50511
[junit4:junit4] 2> 2494348 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2494349 T4654 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2494349 T4654 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158
[junit4:junit4] 2> 2494350 T4654 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/solr.xml
[junit4:junit4] 2> 2494351 T4654 oasc.CoreContainer.<init> New CoreContainer 1656368395
[junit4:junit4] 2> 2494352 T4654 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/'
[junit4:junit4] 2> 2494352 T4654 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/'
[junit4:junit4] 2> 2494537 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158
[junit4:junit4] 2> 2494539 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/collection1
[junit4:junit4] 2> 2494541 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/collection1/conf
[junit4:junit4] 2> 2494747 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/collection1/conf/xslt
[junit4:junit4] 2> 2494752 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/collection1/lib
[junit4:junit4] 2> 2494754 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/collection1/lib/classes
[junit4:junit4] 2> 2494757 T4654 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/conf/core.properties, adding to cores
[junit4:junit4] 2> 2494774 T4654 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2494775 T4654 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2494776 T4654 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2494776 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2494777 T4654 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2494777 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2494778 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2494778 T4654 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2494779 T4654 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2494781 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2494821 T4654 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2494822 T4654 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50508/solr
[junit4:junit4] 2> 2494822 T4654 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2494825 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2494832 T4674 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e0b2e54 name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2494832 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2494838 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2494862 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2494866 T4676 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28d8a549 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2494867 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2494871 T4654 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 2494890 T4654 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 2494901 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 2494907 T4654 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50511_fd_
[junit4:junit4] 2> 2494910 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50511_fd_
[junit4:junit4] 2> 2494918 T4654 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 2494933 T4654 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 2494941 T4654 oasc.Overseer.start Overseer (id=89907761897668611-127.0.0.1:50511_fd_-n_0000000000) starting
[junit4:junit4] 2> 2494951 T4654 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 2494963 T4678 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 2494967 T4654 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 2494980 T4654 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 2494986 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2495004 T4677 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 2495009 T4654 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2495009 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2495010 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2495030 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2495036 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2495045 T4681 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2770ff06 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2495046 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2495049 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2495057 T4654 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 2495615 T4654 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2495635 T4654 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50515
[junit4:junit4] 2> 2495637 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2495637 T4654 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2495638 T4654 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510
[junit4:junit4] 2> 2495638 T4654 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/solr.xml
[junit4:junit4] 2> 2495639 T4654 oasc.CoreContainer.<init> New CoreContainer 2095092861
[junit4:junit4] 2> 2495640 T4654 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/'
[junit4:junit4] 2> 2495640 T4654 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/'
[junit4:junit4] 2> 2495933 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510
[junit4:junit4] 2> 2495935 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/collection1
[junit4:junit4] 2> 2495938 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/collection1/conf
[junit4:junit4] 2> 2496178 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/collection1/conf/xslt
[junit4:junit4] 2> 2496185 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/collection1/lib
[junit4:junit4] 2> 2496188 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/collection1/lib/classes
[junit4:junit4] 2> 2496192 T4654 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/conf/core.properties, adding to cores
[junit4:junit4] 2> 2496223 T4654 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2496224 T4654 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2496225 T4654 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2496225 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2496226 T4654 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2496227 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2496227 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2496228 T4654 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2496229 T4654 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2496229 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2496265 T4654 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2496266 T4654 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50508/solr
[junit4:junit4] 2> 2496267 T4654 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2496274 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2496276 T4692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e7df6e1 name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2496277 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2496283 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2496314 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2496329 T4694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c40753a name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2496330 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2496346 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2497365 T4654 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50515_fd_
[junit4:junit4] 2> 2497369 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50515_fd_
[junit4:junit4] 2> 2497385 T4681 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2497385 T4694 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2497392 T4676 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2497405 T4654 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2497406 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2497406 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2498030 T4654 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2498040 T4654 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50518
[junit4:junit4] 2> 2498041 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2498042 T4654 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2498042 T4654 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879
[junit4:junit4] 2> 2498043 T4654 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/solr.xml
[junit4:junit4] 2> 2498044 T4654 oasc.CoreContainer.<init> New CoreContainer 1995905904
[junit4:junit4] 2> 2498045 T4654 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/'
[junit4:junit4] 2> 2498045 T4654 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/'
[junit4:junit4] 2> 2498256 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879
[junit4:junit4] 2> 2498258 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/collection1
[junit4:junit4] 2> 2498260 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/collection1/conf
[junit4:junit4] 2> 2498450 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/collection1/conf/xslt
[junit4:junit4] 2> 2498461 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/collection1/lib
[junit4:junit4] 2> 2498464 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/collection1/lib/classes
[junit4:junit4] 2> 2498467 T4654 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/conf/core.properties, adding to cores
[junit4:junit4] 2> 2498484 T4654 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2498484 T4654 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2498485 T4654 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2498486 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2498486 T4654 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2498487 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2498487 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2498488 T4654 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2498489 T4654 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2498489 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2498530 T4654 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2498531 T4654 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50508/solr
[junit4:junit4] 2> 2498531 T4654 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2498534 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2498545 T4706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@363cb4c4 name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2498546 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2498553 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2498576 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2498584 T4708 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f575e9c name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2498585 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2498599 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2499613 T4654 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50518_fd_
[junit4:junit4] 2> 2499616 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50518_fd_
[junit4:junit4] 2> 2499630 T4681 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2499631 T4708 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2499631 T4694 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2499632 T4676 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2499644 T4654 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2499647 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2499650 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2500368 T4654 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2500380 T4654 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50521
[junit4:junit4] 2> 2500386 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2500387 T4654 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2500388 T4654 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135
[junit4:junit4] 2> 2500388 T4654 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/solr.xml
[junit4:junit4] 2> 2500389 T4654 oasc.CoreContainer.<init> New CoreContainer 1241711792
[junit4:junit4] 2> 2500389 T4654 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/'
[junit4:junit4] 2> 2500390 T4654 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/'
[junit4:junit4] 2> 2500571 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135
[junit4:junit4] 2> 2500573 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/collection1
[junit4:junit4] 2> 2500575 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/collection1/conf
[junit4:junit4] 2> 2500798 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/collection1/conf/xslt
[junit4:junit4] 2> 2500805 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/collection1/lib
[junit4:junit4] 2> 2500808 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/collection1/lib/classes
[junit4:junit4] 2> 2500812 T4654 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/conf/core.properties, adding to cores
[junit4:junit4] 2> 2500834 T4654 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2500835 T4654 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2500836 T4654 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2500837 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2500838 T4654 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2500839 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2500840 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2500841 T4654 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2500842 T4654 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2500843 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2500895 T4654 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2500896 T4654 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50508/solr
[junit4:junit4] 2> 2500897 T4654 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2500900 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2500911 T4720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78e4536d name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2500912 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2500921 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2500956 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2500963 T4722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7bcdf9f3 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2500964 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2500983 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2501994 T4654 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50521_fd_
[junit4:junit4] 2> 2501999 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50521_fd_
[junit4:junit4] 2> 2502011 T4722 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2502013 T4681 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2502014 T4694 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2502015 T4708 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2502016 T4676 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2502075 T4654 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2502076 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2502077 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2502657 T4654 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2502684 T4654 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50524
[junit4:junit4] 2> 2502685 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2502686 T4654 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2502686 T4654 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550
[junit4:junit4] 2> 2502687 T4654 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/solr.xml
[junit4:junit4] 2> 2502688 T4654 oasc.CoreContainer.<init> New CoreContainer 2136047300
[junit4:junit4] 2> 2502689 T4654 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/'
[junit4:junit4] 2> 2502698 T4654 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/'
[junit4:junit4] 2> 2503067 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550
[junit4:junit4] 2> 2503069 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/collection1
[junit4:junit4] 2> 2503072 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/collection1/conf
[junit4:junit4] 2> 2503363 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/collection1/conf/xslt
[junit4:junit4] 2> 2503376 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/collection1/lib
[junit4:junit4] 2> 2503381 T4654 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/collection1/lib/classes
[junit4:junit4] 2> 2503386 T4654 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/conf/core.properties, adding to cores
[junit4:junit4] 2> 2503406 T4654 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2503406 T4654 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2503407 T4654 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2503407 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2503408 T4654 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2503409 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2503409 T4654 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2503410 T4654 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2503411 T4654 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2503411 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2503461 T4654 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2503462 T4654 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50508/solr
[junit4:junit4] 2> 2503463 T4654 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2503465 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2503472 T4734 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b2bad01 name:ZooKeeperConnection Watcher:127.0.0.1:50508 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2503473 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2503477 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2503507 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2503511 T4736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1aa6fbb4 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2503512 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2503529 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2504542 T4654 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50524_fd_
[junit4:junit4] 2> 2504546 T4654 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50524_fd_
[junit4:junit4] 2> 2504578 T4722 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504578 T4708 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504579 T4736 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504580 T4694 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504582 T4676 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504579 T4681 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2504600 T4654 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2504602 T4654 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2504602 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2504634 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2504680 T4688 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 2504687 T4676 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2504696 T4678 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":null,
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"nodes_used_collection",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 2504697 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50518_fd_
[junit4:junit4] 2> 2504698 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50521_fd_
[junit4:junit4] 2> 2504699 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50524_fd_
[junit4:junit4] 2> 2504705 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50515_fd_
[junit4:junit4] 2> 2504711 T4702 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 2> 2504717 T4717 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 2> 2504725 T4732 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 2> 2504725 T4689 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 2> 2505563 T4677 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2505567 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50518_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50518/fd_"}
[junit4:junit4] 2> 2505568 T4677 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4] 2> 2505579 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50521_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50521/fd_"}
[junit4:junit4] 2> 2505590 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50524_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50524/fd_"}
[junit4:junit4] 2> 2505601 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50515_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50515/fd_"}
[junit4:junit4] 2> 2505613 T4694 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505613 T4708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505614 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505614 T4722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505613 T4681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505623 T4676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2505727 T4702 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2505727 T4702 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2505730 T4702 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2505730 T4702 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2505732 T4717 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 2505733 T4717 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2505733 T4732 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2505734 T4732 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2505735 T4689 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 2505735 T4689 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2505734 T4702 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2505737 T4702 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2505738 T4717 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2505738 T4717 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2505740 T4732 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2505740 T4732 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2505743 T4689 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2505743 T4689 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2505751 T4717 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2505752 T4717 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2505753 T4732 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2505754 T4732 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2505759 T4689 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2505761 T4689 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2505765 T4702 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2505776 T4717 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2505777 T4732 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2505780 T4689 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2505782 T4702 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 2> 2505795 T4717 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 2> 2505798 T4732 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 2> 2505798 T4689 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 2> 2506090 T4732 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2506190 T4717 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2506193 T4689 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2506261 T4702 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2506558 T4732 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2506565 T4689 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2506569 T4732 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2506577 T4732 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4] 2> 2506588 T4689 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2506634 T4702 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2506641 T4689 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4] 2> 2506652 T4717 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2506662 T4702 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2506691 T4702 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4] 2> 2506699 T4717 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2506710 T4717 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4] 2> 2511080 T4732 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2511158 T4732 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2511188 T4732 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2511225 T4689 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2511228 T4717 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2511265 T4717 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2511266 T4689 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2511272 T4717 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2511279 T4689 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2511297 T4702 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2511308 T4702 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2511320 T4702 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2511384 T4732 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511392 T4689 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511417 T4732 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511422 T4717 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511424 T4689 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511437 T4717 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511445 T4689 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511451 T4717 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511453 T4732 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511457 T4689 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511458 T4689 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511459 T4689 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511459 T4732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511464 T4732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511466 T4732 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511458 T4717 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511469 T4717 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511470 T4717 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511468 T4689 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511473 T4689 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511474 T4689 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511475 T4689 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 2> 2511476 T4689 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2511476 T4689 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511474 T4732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511479 T4732 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511479 T4717 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511481 T4717 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511481 T4717 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511482 T4717 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 2> 2511482 T4717 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2511480 T4732 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511485 T4732 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 2> 2511486 T4732 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2511487 T4732 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511488 T4732 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 2> 2511485 T4717 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511490 T4717 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 2> 2511490 T4717 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 2> 2511491 T4717 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2511492 T4717 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 2> 2511488 T4689 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 2> 2511498 T4732 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 2> 2511499 T4732 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2511499 T4689 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 2> 2511500 T4689 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2511507 T4689 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 2> 2511510 T4717 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7faa2594 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45bc184a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2511512 T4717 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2511510 T4702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511510 T4732 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 2> 2511521 T4689 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@52323e76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1303dbb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2511522 T4689 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2511535 T4689 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2511536 T4689 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2511538 T4689 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2511539 T4717 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2511539 T4689 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2511539 T4717 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2511558 T4702 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2511560 T4717 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2511565 T4717 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2511567 T4717 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2511567 T4717 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2511568 T4717 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2511569 T4717 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2511570 T4717 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2511560 T4689 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2511574 T4689 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2511575 T4689 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2511576 T4689 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2511577 T4689 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2511595 T4732 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a9bdb48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7134acb2),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2511596 T4732 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2511602 T4732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2511603 T4732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2511604 T4732 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2511606 T4732 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2511607 T4732 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2511608 T4732 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2511609 T4732 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2511609 T4689 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2511611 T4732 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2511613 T4732 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2511613 T4717 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2511625 T4732 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2511648 T4689 oass.SolrIndexSearcher.<init> Opening Searcher@25b70fd5 main
[junit4:junit4] 2> 2511649 T4689 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 2> 2511651 T4689 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2511652 T4689 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2511657 T4742 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@25b70fd5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2511667 T4702 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511668 T4732 oass.SolrIndexSearcher.<init> Opening Searcher@4403a2d5 main
[junit4:junit4] 2> 2511670 T4732 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 2> 2511672 T4732 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2511672 T4732 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2511678 T4743 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@4403a2d5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2511681 T4689 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 2511683 T4689 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:50515/fd_ collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 2511684 T4717 oass.SolrIndexSearcher.<init> Opening Searcher@683a8c82 main
[junit4:junit4] 2> 2511685 T4717 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 2> 2511683 T4702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511689 T4702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511689 T4702 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2511688 T4717 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2511691 T4717 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2511686 T4732 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2511696 T4732 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:50524/fd_ collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 2511690 T4689 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 2511701 T4732 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 2511700 T4744 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@683a8c82 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2511709 T4717 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 2511709 T4717 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:50521/fd_ collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 2511715 T4717 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2511721 T4702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2511722 T4702 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2511723 T4702 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511725 T4702 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 2> 2511738 T4702 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2511741 T4702 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2511745 T4702 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 2> 2511745 T4702 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 2> 2511746 T4702 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2511749 T4702 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 2> 2511773 T4702 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c55b99a lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f8b98),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2511774 T4702 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2511785 T4702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2511786 T4702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2511787 T4702 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2511788 T4702 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2511789 T4702 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2511791 T4702 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2511793 T4702 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2511794 T4702 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2511799 T4702 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2511809 T4717 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2511823 T4689 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2511846 T4717 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179994
[junit4:junit4] 2> 2511849 T4702 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2511854 T4689 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2511854 T4689 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2511855 T4689 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2511856 T4689 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50515/fd_ START replicas=[http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4] 2> 2511857 T4689 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50515/fd_ DONE. We have no versions. sync failed.
[junit4:junit4] 2> 2511858 T4689 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 2511858 T4689 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 2511859 T4689 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2511859 T4689 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C4275 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@3f895b90 url=http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1 node=127.0.0.1:50524_fd_ C4275_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50524_fd_, base_url=http://127.0.0.1:50524/fd_}
[junit4:junit4] 2> 2511878 T4728 C4275 P50524 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp=/fd_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> 2511890 T4702 oass.SolrIndexSearcher.<init> Opening Searcher@22c86606 main
[junit4:junit4] 2> 2511891 T4702 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 2> 2511893 T4702 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2511894 T4702 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2511904 T4746 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@22c86606 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2511929 T4702 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2511931 T4702 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:50518/fd_ collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 2512352 T4717 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2512353 T4717 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2512354 T4717 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2512354 T4717 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50521/fd_ START replicas=[http://127.0.0.1:50518/fd_/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 2> 2512355 T4717 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50521/fd_ DONE. We have no versions. sync failed.
[junit4:junit4] 2> 2512356 T4717 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 2512356 T4717 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 2512356 T4717 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2512357 T4717 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C4276 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1c13d246 url=http://127.0.0.1:50518/fd_/nodes_used_collection_shard1_replica1 node=127.0.0.1:50518_fd_ C4276_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50518_fd_, base_url=http://127.0.0.1:50518/fd_}
[junit4:junit4] 2> 2512407 T4699 C4276 P50518 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/fd_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 2513248 T4677 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2513302 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513303 T4676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513304 T4681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513303 T4694 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513303 T4722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513303 T4708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2513316 T4702 oasc.ZkController.register We are http://127.0.0.1:50518/fd_/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2513317 T4702 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:50518/fd_
[junit4:junit4] 2> 2513317 T4702 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2513318 T4702 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 2513319 T4748 C4276 P50518 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 2513320 T4748 C4276 P50518 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2513323 T4748 C4276 P50518 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4] 2> 2513326 T4702 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2513330 T4748 C4276 P50518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2513332 T4702 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8621
[junit4:junit4] 2> 2513343 T4689 oasc.ZkController.register We are http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2513344 T4689 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:50515/fd_
[junit4:junit4] 2> 2513345 T4689 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2513347 T4689 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4] 2> 2513348 T4717 oasc.ZkController.register We are http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2513349 T4717 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:50521/fd_
[junit4:junit4] 2> 2513350 T4717 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2513351 T4717 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4] 2> 2513355 T4732 oasc.ZkController.register We are http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2513356 T4732 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:50524/fd_
[junit4:junit4] 2> 2513356 T4732 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2513356 T4732 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C4277 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@3f895b90 url=http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1 node=127.0.0.1:50524_fd_ C4277_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50524_fd_, base_url=http://127.0.0.1:50524/fd_}
[junit4:junit4] 2> 2513358 T4749 C4277 P50524 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 2513358 T4749 C4277 P50524 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2513360 T4749 C4277 P50524 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4] 2> 2513363 T4732 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2513366 T4689 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2513367 T4717 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2513369 T4732 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8644
[junit4:junit4] 2> 2513370 T4749 C4277 P50524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2513382 T4717 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8665
[junit4:junit4] 2> 2513383 T4689 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=8658
[junit4:junit4] 2> 2513393 T4678 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 2> 2513422 T4713 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50518_fd__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2513427 T4694 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 2513447 T4688 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=8767
[junit4:junit4] 2> 2513449 T4676 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2513450 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2513456 T4685 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50524_fd__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2513459 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2513465 T4678 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=8621},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=8644},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=8665},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=8658},core=nodes_used_collection_shard2_replica2}}}
[junit4:junit4] 2> 2513470 T4751 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e7b7817 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2513471 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2513477 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2513488 T4654 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2513494 T4654 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2513502 T4753 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5098b091 name:ZooKeeperConnection Watcher:127.0.0.1:50508/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2513503 T4654 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2513507 T4654 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2513558 T4728 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=3&maxShardsPerNode=2&name=awholynewcollection_0&replicationFactor=3&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 2513565 T4676 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2513571 T4678 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "maxShardsPerNode":"2",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_0",
[junit4:junit4] 2> "replicationFactor":"3"}
[junit4:junit4] 2> 2513571 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50511_fd_
[junit4:junit4] 2> 2513572 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50521_fd_
[junit4:junit4] 2> 2513572 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50518_fd_
[junit4:junit4] 2> 2513573 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50515_fd_
[junit4:junit4] 2> 2513573 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50524_fd_
[junit4:junit4] 2> 2513576 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50511_fd_
[junit4:junit4] 2> 2513581 T4729 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=down
[junit4:junit4] 2> 2513584 T4672 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4] 2> 2513579 T4686 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=down
[junit4:junit4] 2> 2513579 T4703 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica3 state=down
[junit4:junit4] 2> 2513579 T4714 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4] 2> 2513603 T4667 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica3 state=down
[junit4:junit4] 2> 2513584 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50521_fd_
[junit4:junit4] 2> 2513607 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50518_fd_
[junit4:junit4] 2> 2513609 T4678 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50515_fd_
[junit4:junit4] 2> 2513616 T4715 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica1 state=down
[junit4:junit4] 2> 2513622 T4704 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica2 state=down
[junit4:junit4] 2> 2513626 T4687 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica3 state=down
[junit4:junit4] 2> 2514902 T4677 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2514905 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50518_fd__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50518_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50518/fd_"}
[junit4:junit4] 2> 2514924 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50515_fd__nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50515_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50515/fd_"}
[junit4:junit4] 2> 2514940 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50521_fd__nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50521_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50521/fd_"}
[junit4:junit4] 2> 2514955 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50524_fd__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50524_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50524/fd_"}
[junit4:junit4] 2> 2514970 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50524_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50524/fd_"}
[junit4:junit4] 2> 2514971 T4677 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 3
[junit4:junit4] 2> 2514987 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50518_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50518/fd_"}
[junit4:junit4] 2> 2515008 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50515_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50515/fd_"}
[junit4:junit4] 2> 2515033 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50511_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50511/fd_"}
[junit4:junit4] 2> 2515045 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50521_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50521/fd_"}
[junit4:junit4] 2> 2515067 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50511_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50511/fd_"}
[junit4:junit4] 2> 2515079 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50521_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50521/fd_"}
[junit4:junit4] 2> 2515095 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50518_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50518/fd_"}
[junit4:junit4] 2> 2515110 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"3",
[junit4:junit4] 2> "shard":"shard3",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard3_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50515_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50515/fd_"}
[junit4:junit4] 2> 2515127 T4681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515128 T4694 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515129 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515129 T4751 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515130 T4722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515129 T4708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515129 T4753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515157 T4676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2515426 T4713 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50518_fd__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2515427 T4713 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50518_fd__nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:50518_fd_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2005
[junit4:junit4] 2> 2515460 T4685 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50524_fd__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2515461 T4685 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50524_fd__nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:50524_fd_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2005
[junit4:junit4] 2> 2515603 T4729 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 2515606 T4729 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515607 T4703 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 2515607 T4703 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515608 T4686 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 2515609 T4686 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515614 T4686 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2515615 T4686 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2515620 T4686 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2515621 T4686 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2515627 T4729 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2515627 T4729 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2515627 T4714 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 2515629 T4714 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515630 T4667 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 2515631 T4667 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515628 T4715 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1
[junit4:junit4] 2> 2515696 T4715 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515628 T4672 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 2515705 T4672 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515715 T4729 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2515627 T4703 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2515718 T4703 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2515716 T4704 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard3_replica2
[junit4:junit4] 2> 2515719 T4704 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515716 T4687 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3
[junit4:junit4] 2> 2515723 T4687 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2515696 T4686 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515726 T4714 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515727 T4714 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515727 T4667 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515729 T4667 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515729 T4672 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515730 T4672 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515729 T4687 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515732 T4687 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515731 T4704 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515734 T4704 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515733 T4715 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2515736 T4715 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515739 T4703 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2515751 T4667 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard2_replica3/'
[junit4:junit4] 2> 2515751 T4672 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 2> 2515755 T4704 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard3_replica2/'
[junit4:junit4] 2> 2515760 T4714 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 2> 2515763 T4686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard2_replica1/'
[junit4:junit4] 2> 2515768 T4715 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/'
[junit4:junit4] 2> 2515781 T4729 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515785 T4687 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/'
[junit4:junit4] 2> 2515789 T4703 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2515814 T4703 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/'
[junit4:junit4] 2> 2515821 T4729 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_0_shard2_replica2/'
[junit4:junit4] 2> 2516435 T4672 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516584 T4687 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516595 T4686 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516657 T4704 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516746 T4715 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516822 T4667 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516914 T4714 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2516933 T4703 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2517001 T4729 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> ASYNC NEW_CORE C4278 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1c13d246 url=http://127.0.0.1:50518/fd_/nodes_used_collection_shard1_replica1 node=127.0.0.1:50518_fd_ C4278_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50518_fd_, base_url=http://127.0.0.1:50518/fd_}
[junit4:junit4] 2> 2517431 T4748 C4278 P50518 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/ core=nodes_used_collection_shard1_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2517431 T4748 C4278 P50518 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:50518/fd_ START replicas=[http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4] 2> 2517432 T4748 C4278 P50518 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2517432 T4748 C4278 P50518 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2517433 T4748 C4278 P50518 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2517433 T4748 C4278 P50518 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2517433 T4748 C4278 P50518 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2517434 T4748 C4278 P50518 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2/. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2517434 T4748 C4278 P50518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C4279 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@3f895b90 url=http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1 node=127.0.0.1:50524_fd_ C4279_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50524_fd_, base_url=http://127.0.0.1:50524/fd_}
[junit4:junit4] 2> 2517463 T4749 C4279 P50524 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2517464 T4749 C4279 P50524 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:50524/fd_ START replicas=[http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 2> 2517465 T4749 C4279 P50524 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2517466 T4749 C4279 P50524 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2517467 T4749 C4279 P50524 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2517467 T4749 C4279 P50524 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2517467 T4749 C4279 P50524 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> ASYNC NEW_CORE C4280 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@3f895b90 url=http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1 node=127.0.0.1:50524_fd_ C4280_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50524_fd_, base_url=http://127.0.0.1:50524/fd_}
[junit4:junit4] 2> 2517535 T4749 C4280 P50524 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2517536 T4749 C4280 P50524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2517472 T4672 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2517560 T4672 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2517644 T4687 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> ASYNC NEW_CORE C4281 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@7d453c09 url=http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2 node=127.0.0.1:50515_fd_ C4281_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50515_fd_, base_url=http://127.0.0.1:50515/fd_, leader=true}
[junit4:junit4] 2> 2517644 T4685 C4281 P50515 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2517662 T4672 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C4282 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1fdd47d url=http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2 node=127.0.0.1:50521_fd_ C4282_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50521_fd_, base_url=http://127.0.0.1:50521/fd_, leader=true}
[junit4:junit4] 2> 2517673 T4718 C4282 P50521 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/fd_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 2517690 T4687 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> ASYNC NEW_CORE C4283 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@1fdd47d url=http://127.0.0.1:50521/fd_/nodes_used_collection_shard1_replica2 node=127.0.0.1:50521_fd_ C4283_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50521_fd_, base_url=http://127.0.0.1:50521/fd_, leader=true}
[junit4:junit4] 2> 2517727 T4713 C4283 P50521 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C4284 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@7d453c09 url=http://127.0.0.1:50515/fd_/nodes_used_collection_shard2_replica2 node=127.0.0.1:50515_fd_ C4284_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50515_fd_, base_url=http://127.0.0.1:50515/fd_, leader=true}
[junit4:junit4] 2> 2517761 T4685 C4284 P50515 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@52323e76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1303dbb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2517764 T4685 C4284 P50515 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2517791 T4685 C4284 P50515 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@52323e76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1303dbb3),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@52323e76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1303dbb3),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2517792 T4685 C4284 P50515 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2517794 T4685 C4284 P50515 oass.SolrIndexSearcher.<init> Opening Searcher@280bd7ea realtime
[junit4:junit4] 2> 2517795 T4685 C4284 P50515 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2517796 T4685 C4284 P50515 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp=/fd_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 152
[junit4:junit4] 2> ASYNC NEW_CORE C4285 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@3f895b90 url=http://127.0.0.1:50524/fd_/nodes_used_collection_shard2_replica1 node=127.0.0.1:50524_fd_ C4285_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50524_fd_, base_url=http://127.0.0.1:50524/fd_}
[junit4:junit4] 2> 2517810 T4749 C4285 P50524 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2517812 T4749 C4285 P50524 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2517813 T4713 C4283 P50521 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7faa2594 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45bc184a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2517818 T4713 C4283 P50521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2517824 T4688 C4284 P50515 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2517826 T4688 C4284 P50515 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fd_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=7
[junit4:junit4] 2> 2517828 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2517828 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2517829 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2517840 T4713 C4283 P50521 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7faa2594 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45bc184a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7faa2594 lockFactory=org.apache.lucene.store.NativeFSLockFactory@45bc184a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2517842 T4713 C4283 P50521 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2517873 T4685 C4284 P50515 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fd_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=35
[junit4:junit4] 2> 2517875 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2517878 T4713 C4283 P50521 oass.SolrIndexSearcher.<init> Opening Searcher@351404bb realtime
[junit4:junit4] 2> 2517879 T4713 C4283 P50521 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2517884 T4713 C4283 P50521 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica2] webapp=/fd_ path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 157
[junit4:junit4] 2> ASYNC NEW_CORE C4286 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@1c13d246 url=http://127.0.0.1:50518/fd_/nodes_used_collection_shard1_replica1 node=127.0.0.1:50518_fd_ C4286_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50518_fd_, base_url=http://127.0.0.1:50518/fd_}
[junit4:junit4] 2> 2517890 T4748 C4286 P50518 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2517891 T4748 C4286 P50518 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2517899 T4687 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica3] Schema name=test
[junit4:junit4] 2> 2517901 T4749 C4285 P50524 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index.20130622084638328
[junit4:junit4] 2> 2517914 T4718 C4283 P50521 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2517925 T4718 C4283 P50521 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/fd_ path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=15
[junit4:junit4] 2> 2517927 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2517927 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2517928 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2517933 T4716 C4283 P50521 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/fd_ path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=0
[junit4:junit4] 2> 2517941 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1aedd45b lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fa9f604) fullCopy=false
[junit4:junit4] 2> 2517989 T4690 C4284 P50515 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fd_ path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2517996 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2518005 T4688 C4284 P50515 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/fd_ path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2518015 T4686 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518023 T4749 C4285 P50524 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2518025 T4686 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518034 T4748 C4286 P50518 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index.20130622084638449
[junit4:junit4] 2> 2518066 T4749 C4285 P50524 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2518067 T4749 C4285 P50524 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2518069 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@367f46f5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e6ce181) fullCopy=false
[junit4:junit4] 2> 2518115 T4717 C4283 P50521 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp=/fd_ path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2518121 T4748 C4286 P50518 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2518129 T4686 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4] 2> 2518140 T4748 C4286 P50518 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2518141 T4748 C4286 P50518 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2518144 T4748 C4286 P50518 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c55b99a lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f8b98),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6c55b99a lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c4f8b98),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2518146 T4749 C4285 P50524 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a9bdb48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7134acb2),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@3a9bdb48 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7134acb2),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2518151 T4748 C4286 P50518 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2518151 T4749 C4285 P50524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2518164 T4748 C4286 P50518 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2518171 T4749 C4285 P50524 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2518211 T4749 C4285 P50524 oass.SolrIndexSearcher.<init> Opening Searcher@33df5e92 main
[junit4:junit4] 2> 2518199 T4748 C4286 P50518 oass.SolrIndexSearcher.<init> Opening Searcher@16f5b2cf main
[junit4:junit4] 2> 2518219 T4743 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@33df5e92 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2518220 T4749 C4285 P50524 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index.20130622084638328 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index.20130622084638328;done=true>>]
[junit4:junit4] 2> 2518221 T4749 C4285 P50524 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index.20130622084638328
[junit4:junit4] 2> 2518221 T4746 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@16f5b2cf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2518231 T4748 C4286 P50518 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index.20130622084638449 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index.20130622084638449;done=true>>]
[junit4:junit4] 2> 2518232 T4748 C4286 P50518 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index.20130622084638449
[junit4:junit4] 2> 2518237 T4715 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518239 T4749 C4285 P50524 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/nodes_used_collection_shard2_replica1/data/index.20130622084638328
[junit4:junit4] 2> 2518239 T4749 C4285 P50524 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2518240 T4749 C4285 P50524 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2518240 T4749 C4285 P50524 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4] 2> 2518241 T4748 C4286 P50518 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/nodes_used_collection_shard1_replica1/data/index.20130622084638449
[junit4:junit4] 2> 2518241 T4748 C4286 P50518 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2518242 T4748 C4286 P50518 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2518242 T4748 C4286 P50518 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4] 2> 2518245 T4749 C4285 P50524 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2518247 T4748 C4286 P50518 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2518252 T4715 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518324 T4677 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2518327 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50524_fd__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50524_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50524/fd_"}
[junit4:junit4] 2> 2518329 T4703 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518344 T4677 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50518_fd__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50518_fd_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50518/fd_"}
[junit4:junit4] 2> 2518350 T4703 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518357 T4729 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518366 T4703 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4] 2> 2518398 T4729 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518366 T4715 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4] 2> 2518465 T4704 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518472 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518473 T4694 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518474 T4708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518475 T4681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518475 T4751 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518477 T4676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518478 T4722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518479 T4753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2518493 T4704 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518510 T4704 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4] 2> 2518520 T4667 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518522 T4729 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4] 2> 2518590 T4667 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518721 T4714 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2518728 T4714 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2518781 T4714 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4] 2> 2518796 T4667 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4] 2> 2527269 T4672 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2527285 T4672 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2527420 T4672 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2527771 T4672 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2527914 T4672 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2527935 T4672 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2527971 T4672 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2527973 T4672 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2527973 T4672 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2527991 T4672 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2527992 T4672 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2527993 T4672 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2527994 T4672 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4] 2> 2527994 T4672 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2527995 T4672 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2528002 T4672 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data
[junit4:junit4] 2> 2528032 T4672 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data/index/
[junit4:junit4] 2> 2528033 T4672 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2528048 T4672 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data/index
[junit4:junit4] 2> 2528053 T4687 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2528091 T4687 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2528165 T4672 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@32cae19e lockFactory=org.apache.lucene.store.NativeFSLockFactory@be272ca),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2528166 T4672 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2528213 T4672 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2528214 T4672 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2528216 T4672 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2528218 T4672 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2528219 T4703 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2528227 T4672 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2528229 T4672 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2528282 T4672 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2528288 T4672 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2528290 T4672 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2528295 T4687 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2528428 T4703 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2528440 T4672 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2528546 T4703 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2528589 T4672 oass.SolrIndexSearcher.<init> Opening Searcher@2a508965 main
[junit4:junit4] 2> 2528597 T4672 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1371883574158/awholynewcollection_0_shard1_replica1/data/tlog
[junit4:junit4] 2> 2528600 T4672 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2528601 T4672 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2528643 T4687 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2528653 T4763 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@2a508965 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2528660 T4672 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 2528661 T4672 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica1 address:http://127.0.0.1:50511/fd_ collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 2> 2528703 T4672 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4] 2> 2528712 T4715 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2528721 T4687 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2528769 T4687 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2528826 T4715 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2528832 T4687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2528833 T4687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2528836 T4687 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2528838 T4715 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2528875 T4687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2528877 T4687 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2528877 T4687 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2528878 T4703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2528878 T4687 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica3] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data/
[junit4:junit4] 2> 2528880 T4687 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2528881 T4687 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2528886 T4687 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data
[junit4:junit4] 2> 2528886 T4687 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data/index/
[junit4:junit4] 2> 2528887 T4687 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica3] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2528888 T4687 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data/index
[junit4:junit4] 2> 2528899 T4687 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@72a47131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a17b557),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2528900 T4687 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2528906 T4687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2528907 T4687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2528907 T4703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2528908 T4672 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2528947 T4672 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=3 found=1 timeoutin=179986
[junit4:junit4] 2> 2528949 T4703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2528953 T4703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2528953 T4703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2528954 T4703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2528997 T4703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2528998 T4703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2528999 T4703 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2529000 T4703 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica3] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data/
[junit4:junit4] 2> 2529000 T4703 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2529002 T4703 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2529004 T4703 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data
[junit4:junit4] 2> 2529005 T4687 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2529024 T4687 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2529026 T4687 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2529027 T4687 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2529029 T4687 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2529033 T4687 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2529035 T4687 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2529055 T4703 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data/index/
[junit4:junit4] 2> 2529057 T4703 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica3] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2529113 T4729 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2529227 T4703 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data/index
[junit4:junit4] 2> 2529236 T4677 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2529237 T4715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2529251 T4715 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2529259 T4708 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529260 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529261 T4694 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529262 T4681 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529263 T4751 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529263 T4676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529264 T4753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529268 T4722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2529287 T4715 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2529306 T4715 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2529307 T4715 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2529308 T4715 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2529311 T4715 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2529312 T4715 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2529313 T4715 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2529314 T4715 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data/
[junit4:junit4] 2> 2529315 T4715 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@72b36168
[junit4:junit4] 2> 2529316 T4715 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2529317 T4715 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data
[junit4:junit4] 2> 2529318 T4715 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data/index/
[junit4:junit4] 2> 2529318 T4715 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2529319 T4715 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data/index
[junit4:junit4] 2> 2529326 T4715 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@30653b38 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33230f7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2529327 T4715 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2529333 T4715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2529333 T4715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2529403 T4715 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2529405 T4715 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2529406 T4715 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2529406 T4715 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2529407 T4715 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2529408 T4715 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2529409 T4715 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2529389 T4704 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2529425 T4667 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2529428 T4686 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2529431 T4687 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2529437 T4703 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5b13d441 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21c2cb0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2529438 T4703 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2529438 T4729 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2529451 T4729 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2529486 T4703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2529487 T4703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2529488 T4703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2529490 T4703 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2529491 T4703 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2529491 T4703 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2529492 T4703 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2529493 T4703 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2529494 T4704 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2529494 T4703 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2529499 T4704 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2529519 T4703 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2529528 T4715 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2529535 T4667 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2529550 T4686 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2529553 T4667 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2529601 T4686 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2529603 T4703 oass.SolrIndexSearcher.<init> Opening Searcher@6dfec9a8 main
[junit4:junit4] 2> 2529605 T4703 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1371883577879/awholynewcollection_0_shard1_replica3/data/tlog
[junit4:junit4] 2> 2529612 T4703 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2529613 T4703 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2529621 T4766 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica3] Registered new searcher Searcher@6dfec9a8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2529644 T4715 oass.SolrIndexSearcher.<init> Opening Searcher@481e408b main
[junit4:junit4] 2> 2529645 T4715 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1371883580135/awholynewcollection_0_shard3_replica1/data/tlog
[junit4:junit4] 2> 2529647 T4715 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2529648 T4715 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2529655 T4687 oass.SolrIndexSearcher.<init> Opening Searcher@63901eff main
[junit4:junit4] 2> 2529656 T4687 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1371883575510/awholynewcollection_0_shard3_replica3/data/tlog
[junit4:junit4] 2> 2529658 T4687 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2529659 T4687 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2529673 T4765 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica1] Registered new searcher Searcher@481e408b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2529681 T4764 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica3] Registered new searcher Searcher@63901eff main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2529668 T4703 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 2529685
[...truncated too long message...]
State.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 2629370 T4654 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 2629375 T4654 oasc.SolrCore.closeSearcher [awholynewcollection_1_shard2_replica2] Closing main searcher on request.
[junit4:junit4] 2> 2629375 T4654 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 2629376 T4654 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data/index [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data/index;done=false>>]
[junit4:junit4] 2> 2629377 T4654 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data/index
[junit4:junit4] 2> 2629377 T4654 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data;done=false>>]
[junit4:junit4] 2> 2629378 T4654 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1371883582550/awholynewcollection_1_shard2_replica2/data
[junit4:junit4] 2> 2629380 T4954 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89907761897668620-127.0.0.1:50524_fd_-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 2629387 T4736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2629387 T4736 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2629388 T4736 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2629403 T4654 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/fd_,null}
[junit4:junit4] 2> 2629492 T4654 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 2629497 T4654 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50508 50508
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=D4559FF12C31427A -Dtests.slow=true -Dtests.locale=pl_PL -Dtests.timezone=Europe/Monaco -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 136s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: collection already exists: awholynewcollection_1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D4559FF12C31427A:55B311E95B6E2246]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:424)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:264)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:313)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1455)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:438)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2629579 T4654 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {}, docValues:{}, sim=DefaultSimilarity, locale=pl_PL, timezone=Europe/Monaco
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=3,free=117550752,total=370655232
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestFastOutputStream, EchoParamsTest, PluginInfoTest, TestTrie, SolrCoreTest, TestSchemaResource, TestCSVResponseWriter, TestPostingsSolrHighlighter, TestOmitPositions, TestPartialUpdateDeduplication, TestConfig, TestReload, TestSolrDeletionPolicy2, ShowFileRequestHandlerTest, TestElisionMultitermQuery, StandardRequestHandlerTest, SpellPossibilityIteratorTest, SimpleFacetsTest, ResourceLoaderTest, QueryResultKeyTest, SpatialFilterTest, FileBasedSpellCheckerTest, FieldMutatingUpdateProcessorTest, DistributedSpellCheckComponentTest, UUIDFieldTest, UpdateRequestProcessorFactoryTest, TestComponentsName, DistributedTermsComponentTest, PrimitiveFieldTypeTest, TestBinaryField, TestBM25SimilarityFactory, TestSolrXmlPersistence, TestJoin, TestSystemIdResolver, TestSolrDeletionPolicy1, ShardRoutingCustomTest, DistanceFunctionTest, BadComponentTest, DOMUtilTest, QueryEqualityTest, ZkCLITest, PathHierarchyTokenizerFactoryTest, RecoveryZkTest, CurrencyFieldOpenExchangeTest, TestLMDirichletSimilarityFactory, SyncSliceTest, TestQueryUtils, HighlighterTest, TestFoldingMultitermQuery, TestFaceting, RequiredFieldsTest, TestGroupingSearch, TestCloudManagedSchemaAddField, TestPropInjectDefaults, TestReversedWildcardFilterFactory, SuggesterTest, UnloadDistributedZkTest, TestSolr4Spatial, TestLazyCores, TestShardHandlerFactory, TestUniqueKeyFieldResource, PreAnalyzedUpdateProcessorTest, MBeansHandlerTest, TestSweetSpotSimilarityFactory, SortByFunctionTest, JSONWriterTest, TestCodecSupport, TermsComponentTest, TestPhraseSuggestions, DefaultValueUpdateProcessorTest, SchemaVersionSpecificBehaviorTest, SolrTestCaseJ4Test, SystemInfoHandlerTest, TestReplicationHandler, TestExtendedDismaxParser, UniqFieldsUpdateProcessorFactoryTest, CoreAdminHandlerTest, TestAnalyzedSuggestions, TestPseudoReturnFields, LukeRequestHandlerTest, TestRangeQuery, XsltUpdateRequestHandlerTest, SpellingQueryConverterTest, DirectUpdateHandlerTest, SoftAutoCommitTest, ShardSplitTest, TestBadConfig, BadIndexSchemaTest, ShardRoutingTest, TestSerializedLuceneMatchVersion, TestIndexingPerformance, StatsComponentTest, TestSolrQueryParserResource, TestFastLRUCache, DocumentBuilderTest, TestSchemaVersionResource, BadCopyFieldTest, SolrCoreCheckLockOnStartupTest, TestMaxScoreQueryParser, OpenCloseCoreStressTest, TestJmxMonitoredMap, TestSolrJ, TestSchemaSimilarityResource, LeaderElectionTest, ClusterStateTest, OpenExchangeRatesOrgProviderTest, CacheHeaderTest, TestAtomicUpdateErrorCases, TestRealTimeGet, DirectUpdateHandlerOptimizeTest, TestManagedSchema, TestDFRSimilarityFactory, RegexBoostProcessorTest, TestRandomDVFaceting, TestSearchPerf, TestIBSimilarityFactory, TestLRUCache, HighlighterConfigTest, StatelessScriptUpdateProcessorFactoryTest, TestIndexSearcher, TestSort, TestSolrXMLSerializer, TestDocSet, LeaderElectionIntegrationTest, TestBinaryResponseWriter, TestRemoteStreaming, TestSchemaNameResource, DocValuesMultiTest, ChaosMonkeyNothingIsSafeTest, TermVectorComponentDistributedTest, TestFieldResource, QueryElevationComponentTest, TestSuggestSpellingConverter, MinimalSchemaTest, NumericFieldsTest, TestCollationField, SuggesterFSTTest, BasicFunctionalityTest, AnalysisAfterCoreReloadTest, IndexBasedSpellCheckerTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 136.26s, 1 test, 1 error <<< FAILURES!
[...truncated 448 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 error, 18 ignored (6 assumptions)
Total time: 115 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 584 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/584/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:51173/uu_qpg/tw/collection1lastClient and got 267 from http://127.0.0.1:51180/uu_qpg/tw/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51173/uu_qpg/tw/collection1lastClient and got 267 from http://127.0.0.1:51180/uu_qpg/tw/collection1
at __randomizedtesting.SeedInfo.seed([74D793ECD44EB334:F5311DF4A311D308]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9557 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 2594574 T4868 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /uu_qpg/tw
[junit4:junit4] 2> 2594586 T4868 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371860022221
[junit4:junit4] 2> 2594588 T4868 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 2594590 T4869 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 2594691 T4868 oasc.ZkTestServer.run start zk server on port:51166
[junit4:junit4] 2> 2594698 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2594725 T4875 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9df54f name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2594726 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2594728 T4868 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 2594742 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2594750 T4877 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46273ca7 name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2594751 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2594752 T4868 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 2594761 T4868 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 2594771 T4868 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 2594779 T4868 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 2594797 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2594798 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2594813 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 2594814 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 2594823 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2594825 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2594835 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 2594836 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 2594847 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 2594848 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 2594858 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2594860 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2594875 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2594877 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2594893 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2594895 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2594907 T4868 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2594909 T4868 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2595548 T4868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2595572 T4868 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51169
[junit4:junit4] 2> 2595574 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2595574 T4868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2595575 T4868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555
[junit4:junit4] 2> 2595576 T4868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/solr.xml
[junit4:junit4] 2> 2595577 T4868 oasc.CoreContainer.<init> New CoreContainer 1889796430
[junit4:junit4] 2> 2595578 T4868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/'
[junit4:junit4] 2> 2595578 T4868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/'
[junit4:junit4] 2> 2595796 T4868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2595797 T4868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2595797 T4868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2595798 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2595799 T4868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2595799 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2595800 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2595800 T4868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2595801 T4868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2595802 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2595833 T4868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2595834 T4868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51166/solr
[junit4:junit4] 2> 2595835 T4868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2595838 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2595844 T4888 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a0772d4 name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2595845 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2595850 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2595880 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2595885 T4890 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56aaad11 name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2595885 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2595894 T4868 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 2595905 T4868 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 2595913 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 2595918 T4868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51169_uu_qpg%2Ftw
[junit4:junit4] 2> 2595921 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51169_uu_qpg%2Ftw
[junit4:junit4] 2> 2595938 T4868 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 2595952 T4868 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 2595960 T4868 oasc.Overseer.start Overseer (id=89906218416930819-127.0.0.1:51169_uu_qpg%2Ftw-n_0000000000) starting
[junit4:junit4] 2> 2595970 T4868 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 2595985 T4891 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 2595991 T4892 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 2595995 T4868 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 2596003 T4868 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 2596008 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2596023 T4893 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2596023 T4893 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2597500 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2597501 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51169_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51169/uu_qpg/tw"}
[junit4:junit4] 2> 2597502 T4891 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 2597502 T4891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2597515 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2598028 T4893 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/collection1
[junit4:junit4] 2> 2598028 T4893 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 2598030 T4893 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2598031 T4893 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 2598034 T4893 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/collection1/'
[junit4:junit4] 2> 2598036 T4893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2598037 T4893 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/collection1/lib/README' to classloader
[junit4:junit4] 2> 2598113 T4893 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2598221 T4893 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2598225 T4893 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2598234 T4893 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2599360 T4893 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2599377 T4893 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2599383 T4893 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2599428 T4893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2599440 T4893 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2599448 T4893 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2599452 T4893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2599452 T4893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2599452 T4893 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2599456 T4893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2599456 T4893 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2599457 T4893 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2599520 T4893 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371860022555/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/
[junit4:junit4] 2> 2599521 T4893 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d39f05e
[junit4:junit4] 2> 2599522 T4893 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2599524 T4893 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data
[junit4:junit4] 2> 2599525 T4893 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index/
[junit4:junit4] 2> 2599526 T4893 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2599529 T4893 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index
[junit4:junit4] 2> 2599539 T4893 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2599541 T4893 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2599547 T4893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2599548 T4893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2599550 T4893 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2599551 T4893 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2599552 T4893 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2599553 T4893 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2599609 T4893 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2599610 T4893 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2599611 T4893 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2599642 T4893 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2599663 T4893 oass.SolrIndexSearcher.<init> Opening Searcher@26d94794 main
[junit4:junit4] 2> 2599672 T4893 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2599673 T4893 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2599680 T4894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@26d94794 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2599687 T4893 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2599688 T4893 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51169/uu_qpg/tw collection:control_collection shard:shard1
[junit4:junit4] 2> 2599690 T4893 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2599711 T4893 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2599719 T4893 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2599719 T4893 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2599720 T4893 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51169/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2599720 T4893 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2599720 T4893 oasc.SyncStrategy.syncToMe http://127.0.0.1:51169/uu_qpg/tw/collection1/ has no replicas
[junit4:junit4] 2> 2599721 T4893 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51169/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2599721 T4893 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2600537 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2600574 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2600622 T4893 oasc.ZkController.register We are http://127.0.0.1:51169/uu_qpg/tw/collection1/ and leader is http://127.0.0.1:51169/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2600623 T4893 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51169/uu_qpg/tw
[junit4:junit4] 2> 2600623 T4893 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2600623 T4893 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2600624 T4893 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2600627 T4893 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2600631 T4868 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2600631 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2600632 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2600663 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2600668 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2600673 T4897 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ee7bbb5 name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2600673 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2600677 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2600683 T4868 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 2601247 T4868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2601257 T4868 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51173
[junit4:junit4] 2> 2601260 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2601265 T4868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2601266 T4868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317
[junit4:junit4] 2> 2601267 T4868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/solr.xml
[junit4:junit4] 2> 2601268 T4868 oasc.CoreContainer.<init> New CoreContainer 151323966
[junit4:junit4] 2> 2601268 T4868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/'
[junit4:junit4] 2> 2601269 T4868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/'
[junit4:junit4] 2> 2601462 T4868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2601463 T4868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2601464 T4868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2601465 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2601465 T4868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2601466 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2601466 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2601467 T4868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2601468 T4868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2601468 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2601496 T4868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2601497 T4868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51166/solr
[junit4:junit4] 2> 2601498 T4868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2601500 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2601505 T4908 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f2629d2 name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2601506 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2601510 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2601540 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2601545 T4910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9c1468a name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2601545 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2601557 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2602098 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2602100 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51169_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51169_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51169/uu_qpg/tw"}
[junit4:junit4] 2> 2602117 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2602119 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2602120 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2602569 T4868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51173_uu_qpg%2Ftw
[junit4:junit4] 2> 2602572 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51173_uu_qpg%2Ftw
[junit4:junit4] 2> 2602579 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2602581 T4910 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2602581 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2602581 T4890 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2602582 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2602605 T4897 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2602609 T4911 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2602610 T4911 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2603639 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2603641 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51173_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51173/uu_qpg/tw"}
[junit4:junit4] 2> 2603642 T4891 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 2603642 T4891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2603682 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2603682 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2603682 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2604615 T4911 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/collection1
[junit4:junit4] 2> 2604615 T4911 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2604618 T4911 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2604618 T4911 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2604623 T4911 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/collection1/'
[junit4:junit4] 2> 2604626 T4911 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2604627 T4911 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/collection1/lib/README' to classloader
[junit4:junit4] 2> 2604753 T4911 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2604914 T4911 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2604918 T4911 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2604931 T4911 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2606268 T4911 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2606286 T4911 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2606292 T4911 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2606333 T4911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2606348 T4911 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2606364 T4911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2606369 T4911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2606370 T4911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2606371 T4911 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2606375 T4911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2606376 T4911 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2606376 T4911 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2606377 T4911 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371860028317/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/
[junit4:junit4] 2> 2606378 T4911 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d39f05e
[junit4:junit4] 2> 2606379 T4911 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2606381 T4911 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1
[junit4:junit4] 2> 2606383 T4911 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index/
[junit4:junit4] 2> 2606385 T4911 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2606387 T4911 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index
[junit4:junit4] 2> 2606404 T4911 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2606409 T4911 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2606416 T4911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2606417 T4911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2606418 T4911 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2606419 T4911 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2606420 T4911 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2606420 T4911 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2606421 T4911 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2606424 T4911 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2606426 T4911 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2606447 T4911 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2606459 T4911 oass.SolrIndexSearcher.<init> Opening Searcher@3250f035 main
[junit4:junit4] 2> 2606461 T4911 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2606461 T4911 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2606470 T4912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3250f035 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2606476 T4911 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2606477 T4911 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51173/uu_qpg/tw collection:collection1 shard:shard1
[junit4:junit4] 2> 2606480 T4911 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 2606510 T4911 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2606517 T4911 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2606518 T4911 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2606518 T4911 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2606519 T4911 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2606519 T4911 oasc.SyncStrategy.syncToMe http://127.0.0.1:51173/uu_qpg/tw/collection1/ has no replicas
[junit4:junit4] 2> 2606519 T4911 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2606520 T4911 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 2606718 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2606748 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2606748 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2606749 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2606796 T4911 oasc.ZkController.register We are http://127.0.0.1:51173/uu_qpg/tw/collection1/ and leader is http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2606797 T4911 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51173/uu_qpg/tw
[junit4:junit4] 2> 2606797 T4911 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2606798 T4911 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2606798 T4911 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2606805 T4911 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2606808 T4868 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2606809 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2606810 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2607485 T4868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2607497 T4868 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51176
[junit4:junit4] 2> 2607498 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2607498 T4868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2607499 T4868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474
[junit4:junit4] 2> 2607500 T4868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/solr.xml
[junit4:junit4] 2> 2607501 T4868 oasc.CoreContainer.<init> New CoreContainer 1101748387
[junit4:junit4] 2> 2607501 T4868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/'
[junit4:junit4] 2> 2607502 T4868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/'
[junit4:junit4] 2> 2607760 T4868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2607761 T4868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2607766 T4868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2607770 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2607771 T4868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2607771 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2607772 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2607773 T4868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2607774 T4868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2607775 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2607806 T4868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2607807 T4868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51166/solr
[junit4:junit4] 2> 2607807 T4868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2607810 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2607815 T4924 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60148266 name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2607816 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2607821 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2607850 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2607856 T4926 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1936cc8c name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2607857 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2607868 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2608272 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2608273 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51173_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51173_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51173/uu_qpg/tw"}
[junit4:junit4] 2> 2608284 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608285 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608286 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608285 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608879 T4868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51176_uu_qpg%2Ftw
[junit4:junit4] 2> 2608883 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51176_uu_qpg%2Ftw
[junit4:junit4] 2> 2608891 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608897 T4890 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2608897 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2608898 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 2608911 T4926 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2608916 T4910 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2608916 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2608924 T4897 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2608930 T4927 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2608930 T4927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2609803 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2609805 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51176_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51176/uu_qpg/tw"}
[junit4:junit4] 2> 2609805 T4891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2609805 T4891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2609816 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2609817 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2609824 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2609825 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2609939 T4927 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/collection1
[junit4:junit4] 2> 2609939 T4927 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2609941 T4927 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2609942 T4927 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2609945 T4927 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/collection1/'
[junit4:junit4] 2> 2609947 T4927 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2609948 T4927 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/collection1/lib/README' to classloader
[junit4:junit4] 2> 2610045 T4927 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2610170 T4927 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2610174 T4927 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2610184 T4927 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 2611448 T4927 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2611470 T4927 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2611477 T4927 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2611513 T4927 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2611523 T4927 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2611532 T4927 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2611536 T4927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2611537 T4927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2611538 T4927 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2611547 T4927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2611547 T4927 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2611548 T4927 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2611548 T4927 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371860034474/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/
[junit4:junit4] 2> 2611549 T4927 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d39f05e
[junit4:junit4] 2> 2611550 T4927 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2611551 T4927 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2
[junit4:junit4] 2> 2611552 T4927 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index/
[junit4:junit4] 2> 2611553 T4927 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2611554 T4927 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index
[junit4:junit4] 2> 2611563 T4927 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2611564 T4927 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2611570 T4927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2611570 T4927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2611571 T4927 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2611573 T4927 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2611574 T4927 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2611574 T4927 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2611575 T4927 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2611576 T4927 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2611577 T4927 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2611608 T4927 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2611629 T4927 oass.SolrIndexSearcher.<init> Opening Searcher@56deaa83 main
[junit4:junit4] 2> 2611631 T4927 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2611631 T4927 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2611637 T4928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56deaa83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2611642 T4927 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2611642 T4927 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51176/uu_qpg/tw collection:collection1 shard:shard1
[junit4:junit4] 2> 2611653 T4927 oasc.ZkController.register We are http://127.0.0.1:51176/uu_qpg/tw/collection1/ and leader is http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2611653 T4927 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51176/uu_qpg/tw
[junit4:junit4] 2> 2611654 T4927 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2611654 T4927 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1259 name=collection1 org.apache.solr.core.SolrCore@52036f9 url=http://127.0.0.1:51176/uu_qpg/tw/collection1 node=127.0.0.1:51176_uu_qpg%2Ftw C1259_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51176_uu_qpg%2Ftw, base_url=http://127.0.0.1:51176/uu_qpg/tw}
[junit4:junit4] 2> 2611655 T4929 C1259 P51176 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2611656 T4929 C1259 P51176 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2611656 T4929 C1259 P51176 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2611656 T4929 C1259 P51176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2611660 T4929 C1259 P51176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2611673 T4927 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2611684 T4868 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2611685 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2611685 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2611697 T4902 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51176_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2612262 T4868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2612272 T4868 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51180
[junit4:junit4] 2> 2612274 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2612274 T4868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2612275 T4868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344
[junit4:junit4] 2> 2612275 T4868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/solr.xml
[junit4:junit4] 2> 2612276 T4868 oasc.CoreContainer.<init> New CoreContainer 651322997
[junit4:junit4] 2> 2612277 T4868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/'
[junit4:junit4] 2> 2612277 T4868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/'
[junit4:junit4] 2> 2612550 T4868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2612551 T4868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2612551 T4868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2612552 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2612554 T4868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2612554 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2612555 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2612556 T4868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2612557 T4868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2612558 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2612586 T4868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2612587 T4868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51166/solr
[junit4:junit4] 2> 2612588 T4868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2612592 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2612597 T4941 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@222cc3c7 name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2612598 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2612609 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2612637 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2612642 T4943 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56079189 name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2612642 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2612654 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2612842 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2612844 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51176_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51176_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51176/uu_qpg/tw"}
[junit4:junit4] 2> 2612862 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2612862 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2612863 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2612863 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2612863 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2613664 T4868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51180_uu_qpg%2Ftw
[junit4:junit4] 2> 2613667 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51180_uu_qpg%2Ftw
[junit4:junit4] 2> 2613674 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2613674 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 2613678 T4890 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2613678 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2613679 T4943 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2613679 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2613682 T4910 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2613682 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2613692 T4926 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2613693 T4897 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2613701 T4902 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51176_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2613702 T4902 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51176_uu_qpg%252Ftw_collection1&state=recovering&nodeName=127.0.0.1:51176_uu_qpg%252Ftw&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2005
[junit4:junit4] 2> 2613716 T4944 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2613717 T4944 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2614381 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2614383 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51180_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51180/uu_qpg/tw"}
[junit4:junit4] 2> 2614384 T4891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2614384 T4891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2614395 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2614396 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2614396 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2614397 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2614409 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2614721 T4944 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/collection1
[junit4:junit4] 2> 2614722 T4944 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2614723 T4944 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2614724 T4944 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2614728 T4944 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/collection1/'
[junit4:junit4] 2> 2614730 T4944 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2614730 T4944 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/collection1/lib/README' to classloader
[junit4:junit4] 2> 2614829 T4944 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2614937 T4944 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2614941 T4944 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2614948 T4944 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1260 name=collection1 org.apache.solr.core.SolrCore@52036f9 url=http://127.0.0.1:51176/uu_qpg/tw/collection1 node=127.0.0.1:51176_uu_qpg%2Ftw C1260_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51176_uu_qpg%2Ftw, base_url=http://127.0.0.1:51176/uu_qpg/tw}
[junit4:junit4] 2> 2615706 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51173/uu_qpg/tw/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2615707 T4929 C1260 P51176 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51176/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2615707 T4929 C1260 P51176 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2615728 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2615732 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2615733 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2615733 T4929 C1260 P51176 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2615733 T4929 C1260 P51176 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51173/uu_qpg/tw/collection1/. core=collection1
[junit4:junit4] 2> 2615734 T4929 C1260 P51176 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1261 name=collection1 org.apache.solr.core.SolrCore@45157ec4 url=http://127.0.0.1:51173/uu_qpg/tw/collection1 node=127.0.0.1:51173_uu_qpg%2Ftw C1261_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51173_uu_qpg%2Ftw, base_url=http://127.0.0.1:51173/uu_qpg/tw, leader=true}
[junit4:junit4] 2> 2615748 T4901 C1261 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2615800 T4903 C1261 P51173 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2615815 T4903 C1261 P51173 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2615817 T4903 C1261 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2615824 T4903 C1261 P51173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2615825 T4903 C1261 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2615827 T4903 C1261 P51173 oass.SolrIndexSearcher.<init> Opening Searcher@3d8e89ee realtime
[junit4:junit4] 2> 2615828 T4903 C1261 P51173 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2615828 T4903 C1261 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4] 2> 2615832 T4929 C1260 P51176 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2615832 T4929 C1260 P51176 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2615838 T4905 C1261 P51173 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2615839 T4905 C1261 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3
[junit4:junit4] 2> 2615840 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2615840 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2615841 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2615846 T4906 C1261 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 2615857 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2615866 T4929 C1260 P51176 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491
[junit4:junit4] 2> 2615868 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491 lockFactory=org.apache.lucene.store.NativeFSLockFactory@76089983; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2615876 T4902 C1261 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 2615885 T4929 C1260 P51176 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2615890 T4929 C1260 P51176 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2615890 T4929 C1260 P51176 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2615897 T4929 C1260 P51176 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2615899 T4929 C1260 P51176 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2615900 T4929 C1260 P51176 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2615901 T4929 C1260 P51176 oass.SolrIndexSearcher.<init> Opening Searcher@3100397 main
[junit4:junit4] 2> 2615903 T4928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3100397 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2615905 T4929 C1260 P51176 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491;done=true>>]
[junit4:junit4] 2> 2615905 T4929 C1260 P51176 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491
[junit4:junit4] 2> 2615905 T4929 C1260 P51176 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index.20130621231403491
[junit4:junit4] 2> 2615907 T4929 C1260 P51176 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2615907 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2615908 T4929 C1260 P51176 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2615908 T4929 C1260 P51176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2615911 T4929 C1260 P51176 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2615937 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2615940 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51176_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51176_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51176/uu_qpg/tw"}
[junit4:junit4] 2> 2615986 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2615987 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2615987 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2615988 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2615987 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2616188 T4944 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2616211 T4944 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2616216 T4944 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2616249 T4944 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2616280 T4944 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2616289 T4944 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2616293 T4944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2616294 T4944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2616294 T4944 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2616297 T4944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2616298 T4944 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2616298 T4944 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2616300 T4944 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371860039344/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/
[junit4:junit4] 2> 2616300 T4944 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d39f05e
[junit4:junit4] 2> 2616301 T4944 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2616302 T4944 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3
[junit4:junit4] 2> 2616303 T4944 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index/
[junit4:junit4] 2> 2616304 T4944 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2616305 T4944 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index
[junit4:junit4] 2> 2616314 T4944 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2616314 T4944 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2616319 T4944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2616319 T4944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2616320 T4944 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2616322 T4944 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2616323 T4944 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2616324 T4944 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2616334 T4944 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2616336 T4944 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2616336 T4944 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2616358 T4944 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2616376 T4944 oass.SolrIndexSearcher.<init> Opening Searcher@652206 main
[junit4:junit4] 2> 2616378 T4944 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2616379 T4944 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2616385 T4947 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@652206 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2616389 T4944 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2616389 T4944 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51180/uu_qpg/tw collection:collection1 shard:shard1
[junit4:junit4] 2> 2616403 T4944 oasc.ZkController.register We are http://127.0.0.1:51180/uu_qpg/tw/collection1/ and leader is http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2616404 T4944 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51180/uu_qpg/tw
[junit4:junit4] 2> 2616404 T4944 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2616405 T4944 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1262 name=collection1 org.apache.solr.core.SolrCore@1f64156 url=http://127.0.0.1:51180/uu_qpg/tw/collection1 node=127.0.0.1:51180_uu_qpg%2Ftw C1262_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51180_uu_qpg%2Ftw, base_url=http://127.0.0.1:51180/uu_qpg/tw}
[junit4:junit4] 2> 2616407 T4948 C1262 P51180 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2616408 T4948 C1262 P51180 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2616408 T4944 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2616408 T4948 C1262 P51180 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2616409 T4948 C1262 P51180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2616411 T4868 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2616412 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2616413 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2616412 T4948 C1262 P51180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2616445 T4903 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51180_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2617047 T4868 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2617061 T4868 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51187
[junit4:junit4] 2> 2617063 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2617063 T4868 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2617064 T4868 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079
[junit4:junit4] 2> 2617065 T4868 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/solr.xml
[junit4:junit4] 2> 2617065 T4868 oasc.CoreContainer.<init> New CoreContainer 490779839
[junit4:junit4] 2> 2617067 T4868 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/'
[junit4:junit4] 2> 2617067 T4868 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/'
[junit4:junit4] 2> 2617327 T4868 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2617328 T4868 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2617329 T4868 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2617329 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2617330 T4868 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2617330 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2617331 T4868 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2617332 T4868 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2617333 T4868 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2617334 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2617361 T4868 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2617362 T4868 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51166/solr
[junit4:junit4] 2> 2617363 T4868 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2617366 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2617371 T4960 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@253d219a name:ZooKeeperConnection Watcher:127.0.0.1:51166 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2617373 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2617378 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2617405 T4868 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2617410 T4962 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53c35d33 name:ZooKeeperConnection Watcher:127.0.0.1:51166/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2617411 T4868 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2617422 T4868 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2617507 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2617509 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51180_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51180_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51180/uu_qpg/tw"}
[junit4:junit4] 2> 2617523 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2617524 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2617525 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2617525 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2617526 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2617526 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2618432 T4868 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51187_uu_qpg%2Ftw
[junit4:junit4] 2> 2618435 T4868 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51187_uu_qpg%2Ftw
[junit4:junit4] 2> 2618442 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2618443 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 2618447 T4890 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618447 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2618450 T4910 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618450 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2618448 T4943 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618453 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2618448 T4903 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51180_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2618455 T4962 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618455 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2618456 T4903 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51180_uu_qpg%252Ftw_collection1&state=recovering&nodeName=127.0.0.1:51180_uu_qpg%252Ftw&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2011
[junit4:junit4] 2> 2618464 T4897 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618468 T4926 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2618482 T4963 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 2618482 T4963 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2619050 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2619052 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51187_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51187/uu_qpg/tw"}
[junit4:junit4] 2> 2619052 T4891 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 2619053 T4891 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 2619065 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619065 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619066 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619066 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619065 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619067 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2619486 T4963 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/collection1
[junit4:junit4] 2> 2619487 T4963 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 2619489 T4963 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2619490 T4963 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 2619495 T4963 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/collection1/'
[junit4:junit4] 2> 2619498 T4963 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 2619499 T4963 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/collection1/lib/README' to classloader
[junit4:junit4] 2> 2619629 T4963 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2619745 T4963 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2619749 T4963 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2619758 T4963 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1263 name=collection1 org.apache.solr.core.SolrCore@1f64156 url=http://127.0.0.1:51180/uu_qpg/tw/collection1 node=127.0.0.1:51180_uu_qpg%2Ftw C1263_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51180_uu_qpg%2Ftw, base_url=http://127.0.0.1:51180/uu_qpg/tw}
[junit4:junit4] 2> 2620462 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51173/uu_qpg/tw/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2620463 T4948 C1263 P51180 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51180/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2620464 T4948 C1263 P51180 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2620465 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2620465 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2620465 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2620467 T4948 C1263 P51180 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2620467 T4948 C1263 P51180 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51173/uu_qpg/tw/collection1/. core=collection1
[junit4:junit4] 2> 2620468 T4948 C1263 P51180 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1264 name=collection1 org.apache.solr.core.SolrCore@45157ec4 url=http://127.0.0.1:51173/uu_qpg/tw/collection1 node=127.0.0.1:51173_uu_qpg%2Ftw C1264_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51173_uu_qpg%2Ftw, base_url=http://127.0.0.1:51173/uu_qpg/tw, leader=true}
[junit4:junit4] 2> 2620469 T4901 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2620517 T4905 C1264 P51173 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2620565 T4905 C1264 P51173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2620567 T4905 C1264 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2620570 T4905 C1264 P51173 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2620571 T4905 C1264 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 54
[junit4:junit4] 2> 2620589 T4948 C1263 P51180 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2620591 T4948 C1263 P51180 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2620594 T4902 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 2620595 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 2620595 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2620596 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2620602 T4901 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1
[junit4:junit4] 2> 2620604 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2620606 T4948 C1263 P51180 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237
[junit4:junit4] 2> 2620608 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79829f1e; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2620613 T4903 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=1
[junit4:junit4] 2> 2620616 T4948 C1263 P51180 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2620628 T4948 C1263 P51180 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2620628 T4948 C1263 P51180 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2620635 T4948 C1263 P51180 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2620690 T4948 C1263 P51180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2620697 T4948 C1263 P51180 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2620699 T4948 C1263 P51180 oass.SolrIndexSearcher.<init> Opening Searcher@46daf00e main
[junit4:junit4] 2> 2620701 T4947 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46daf00e main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 2620703 T4948 C1263 P51180 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237;done=true>>]
[junit4:junit4] 2> 2620704 T4948 C1263 P51180 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237
[junit4:junit4] 2> 2620704 T4948 C1263 P51180 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index.20130621231408237
[junit4:junit4] 2> 2620705 T4948 C1263 P51180 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2620705 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2620706 T4948 C1263 P51180 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2620706 T4948 C1263 P51180 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2620708 T4948 C1263 P51180 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2621242 T4963 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2621268 T4963 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2621274 T4963 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2621316 T4963 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2621354 T4963 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2621366 T4963 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2621370 T4963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2621373 T4963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2621374 T4963 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2621378 T4963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2621379 T4963 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2621379 T4963 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2621380 T4963 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371860044079/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/
[junit4:junit4] 2> 2621380 T4963 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d39f05e
[junit4:junit4] 2> 2621381 T4963 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 2621383 T4963 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4
[junit4:junit4] 2> 2621385 T4963 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index/
[junit4:junit4] 2> 2621386 T4963 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2621388 T4963 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index
[junit4:junit4] 2> 2621404 T4963 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2621404 T4963 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2621411 T4963 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2621413 T4963 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2621414 T4963 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2621416 T4963 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2621418 T4963 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2621419 T4963 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2621425 T4963 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2621427 T4963 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2621427 T4963 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2621451 T4963 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2621479 T4963 oass.SolrIndexSearcher.<init> Opening Searcher@566bd25a main
[junit4:junit4] 2> 2621481 T4963 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2621482 T4963 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2621490 T4966 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@566bd25a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2621497 T4963 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2621498 T4963 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51187/uu_qpg/tw collection:collection1 shard:shard1
[junit4:junit4] 2> 2621516 T4963 oasc.ZkController.register We are http://127.0.0.1:51187/uu_qpg/tw/collection1/ and leader is http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2621518 T4963 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51187/uu_qpg/tw
[junit4:junit4] 2> 2621518 T4963 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 2621519 T4963 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1265 name=collection1 org.apache.solr.core.SolrCore@69917765 url=http://127.0.0.1:51187/uu_qpg/tw/collection1 node=127.0.0.1:51187_uu_qpg%2Ftw C1265_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51187_uu_qpg%2Ftw, base_url=http://127.0.0.1:51187/uu_qpg/tw}
[junit4:junit4] 2> 2621520 T4967 C1265 P51187 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 2621521 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2621521 T4967 C1265 P51187 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 2621522 T4967 C1265 P51187 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2621523 T4963 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2621526 T4967 C1265 P51187 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2621537 T4868 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2621538 T4868 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2621539 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2621568 T4906 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51187_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2621577 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2621588 T4868 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 2621589 T4868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 2621592 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2622099 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2622101 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51180_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51180_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51180/uu_qpg/tw"}
[junit4:junit4] 2> 2622115 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51187_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51187_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51187/uu_qpg/tw"}
[junit4:junit4] 2> 2622136 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622139 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622140 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622141 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622140 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622139 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2622570 T4906 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51187_uu_qpg%2Ftw_collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 2622572 T4906 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51187_uu_qpg%252Ftw_collection1&state=recovering&nodeName=127.0.0.1:51187_uu_qpg%252Ftw&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> 2622605 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2623610 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1265_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51187_uu_qpg%2Ftw, base_url=http://127.0.0.1:51187/uu_qpg/tw}
[junit4:junit4] 2> 2624575 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51173/uu_qpg/tw/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2624575 T4967 C1265 P51187 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51187/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2624576 T4967 C1265 P51187 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2624576 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 2624577 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 2624577 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 2624577 T4967 C1265 P51187 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2624577 T4967 C1265 P51187 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51173/uu_qpg/tw/collection1/. core=collection1
[junit4:junit4] 2> 2624578 T4967 C1265 P51187 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2624600 T4906 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2624616 T4904 C1264 P51173 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2624624 T4904 C1264 P51173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2624625 T4904 C1264 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2624625 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2624628 T4904 C1264 P51173 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2624630 T4904 C1264 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 2624633 T4967 C1265 P51187 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2624634 T4967 C1265 P51187 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2624639 T4902 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 2624642 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 2624642 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2624642 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2624654 T4901 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=2
[junit4:junit4] 2> 2624657 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2624659 T4967 C1265 P51187 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290
[junit4:junit4] 2> 2624661 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2409a660; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 2624666 T4906 C1264 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 2624670 T4967 C1265 P51187 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2624674 T4967 C1265 P51187 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2624675 T4967 C1265 P51187 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 2624683 T4967 C1265 P51187 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2624685 T4967 C1265 P51187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2624686 T4967 C1265 P51187 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2624688 T4967 C1265 P51187 oass.SolrIndexSearcher.<init> Opening Searcher@1e683c7f main
[junit4:junit4] 2> 2624691 T4966 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e683c7f main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 2624692 T4967 C1265 P51187 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290;done=true>>]
[junit4:junit4] 2> 2624692 T4967 C1265 P51187 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290
[junit4:junit4] 2> 2624693 T4967 C1265 P51187 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index.20130621231412290
[junit4:junit4] 2> 2624695 T4967 C1265 P51187 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 2624695 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 2624696 T4967 C1265 P51187 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 2624696 T4967 C1265 P51187 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 2624701 T4967 C1265 P51187 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 2625202 T4891 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2625205 T4891 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51187_uu_qpg%2Ftw_collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51187_uu_qpg%2Ftw",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51187/uu_qpg/tw"}
[junit4:junit4] 2> 2625242 T4890 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625243 T4910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625258 T4926 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625259 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625259 T4897 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625262 T4943 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2625637 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2625640 T4868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1266 name=collection1 org.apache.solr.core.SolrCore@798ded3f url=http://127.0.0.1:51169/uu_qpg/tw/collection1 node=127.0.0.1:51169_uu_qpg%2Ftw C1266_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51169_uu_qpg%2Ftw, base_url=http://127.0.0.1:51169/uu_qpg/tw, leader=true}
[junit4:junit4] 2> 2625669 T4882 C1266 P51169 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2625680 T4882 C1266 P51169 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2625683 T4882 C1266 P51169 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2625702 T4882 C1266 P51169 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2625703 T4882 C1266 P51169 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2625709 T4882 C1266 P51169 oass.SolrIndexSearcher.<init> Opening Searcher@4397f3ff main
[junit4:junit4] 2> 2625711 T4882 C1266 P51169 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2625712 T4894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4397f3ff main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2625713 T4882 C1266 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 44
[junit4:junit4] 2> 2625739 T4905 C1264 P51173 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51173/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51176/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51180/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51187/uu_qpg/tw/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> ASYNC NEW_CORE C1267 name=collection1 org.apache.solr.core.SolrCore@45157ec4 url=http://127.0.0.1:51173/uu_qpg/tw/collection1 node=127.0.0.1:51173_uu_qpg%2Ftw C1267_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51173_uu_qpg%2Ftw, base_url=http://127.0.0.1:51173/uu_qpg/tw, leader=true}
[junit4:junit4] 2> 2625885 T4902 C1267 P51173 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2625894 T4902 C1267 P51173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2625896 T4902 C1267 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2625899 T4902 C1267 P51173 oass.SolrIndexSearcher.<init> Opening Searcher@31cef2e5 main
[junit4:junit4] 2> 2625903 T4912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@31cef2e5 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2625905 T4902 C1267 P51173 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> ASYNC NEW_CORE C1268 name=collection1 org.apache.solr.core.SolrCore@52036f9 url=http://127.0.0.1:51176/uu_qpg/tw/collection1 node=127.0.0.1:51176_uu_qpg%2Ftw C1268_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51176_uu_qpg%2Ftw, base_url=http://127.0.0.1:51176/uu_qpg/tw}
[junit4:junit4] 2> 2625906 T4920 C1268 P51176 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2625906 T4902 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4] 2> ASYNC NEW_CORE C1269 name=collection1 org.apache.solr.core.SolrCore@1f64156 url=http://127.0.0.1:51180/uu_qpg/tw/collection1 node=127.0.0.1:51180_uu_qpg%2Ftw C1269_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51180_uu_qpg%2Ftw, base_url=http://127.0.0.1:51180/uu_qpg/tw}
[junit4:junit4] 2> 2625917 T4937 C1269 P51180 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C1270 name=collection1 org.apache.solr.core.SolrCore@69917765 url=http://127.0.0.1:51187/uu_qpg/tw/collection1 node=127.0.0.1:51187_uu_qpg%2Ftw C1270_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51187_uu_qpg%2Ftw, base_url=http://127.0.0.1:51187/uu_qpg/tw}
[junit4:junit4] 2> 2625918 T4955 C1270 P51187 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2625930 T4920 C1268 P51176 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2625931 T4920 C1268 P51176 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2625938 T4920 C1268 P51176 oass.SolrIndexSearcher.<init> Opening Searcher@271754f5 main
[junit4:junit4] 2> 2625941 T4928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@271754f5 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2625942 T4920 C1268 P51176 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2625944 T4920 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4] 2> 2625943 T4955 C1270 P51187 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2625948 T4955 C1270 P51187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2625943 T4937 C1269 P51180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2625955 T4937 C1269 P51180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2625971 T4937 C1269 P51180 oass.SolrIndexSearcher.<init> Opening Searcher@888378a main
[junit4:junit4] 2> 2625971 T4955 C1270 P51187 oass.SolrIndexSearcher.<init> Opening Searcher@734cdb85 main
[junit4:junit4] 2> 2625974 T4955 C1270 P51187 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2625975 T4947 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@888378a main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 2625976 T4937 C1269 P51180 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2625976 T4966 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@734cdb85 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 2625977 T4937 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 60
[junit4:junit4] 2> 2625987 T4955 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 69
[junit4:junit4] 2> 2625992 T4905 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 254
[junit4:junit4] 2> 2625998 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2626005 T4906 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 2626011 T4922 C1268 P51176 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 2626020 T4939 C1269 P51180 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 2626028 T4957 C1270 P51187 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> ASYNC NEW_CORE C1271 name=collection1 org.apache.solr.core.SolrCore@798ded3f url=http://127.0.0.1:51169/uu_qpg/tw/collection1 node=127.0.0.1:51169_uu_qpg%2Ftw C1271_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51169_uu_qpg%2Ftw, base_url=http://127.0.0.1:51169/uu_qpg/tw, leader=true}
[junit4:junit4] 2> 2628038 T4882 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438499529732128768)} 0 4
[junit4:junit4] 2> 2628071 T4958 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={update.distrib=FROMLEADER&_version_=-1438499529749954560&update.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438499529749954560)} 0 11
[junit4:junit4] 2> 2628080 T4939 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={update.distrib=FROMLEADER&_version_=-1438499529749954560&update.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438499529749954560)} 0 12
[junit4:junit4] 2> 2628082 T4922 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={update.distrib=FROMLEADER&_version_=-1438499529749954560&update.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438499529749954560)} 0 11
[junit4:junit4] 2> 2628087 T4906 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438499529749954560)} 0 35
[junit4:junit4] 2> 2628089 T4957 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 46
[junit4:junit4] 2> 2628104 T4883 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[0 (1438499529792946176)]} 0 11
[junit4:junit4] 2> 2628130 T4935 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438499529819160576)]} 0 3
[junit4:junit4] 2> 2628137 T4953 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438499529819160576)]} 0 3
[junit4:junit4] 2> 2628138 T4917 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438499529819160576)]} 0 6
[junit4:junit4] 2> 2628140 T4903 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51180/uu_qpg/tw/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1438499529819160576)]} 0 22
[junit4:junit4] 2> 2628141 T4934 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[0]} 0 30
[junit4:junit4] 2> 2628152 T4884 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[1 (1438499529851666432)]} 0 4
[junit4:junit4] 2> 2628167 T4954 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438499529864249344)]} 0 1
[junit4:junit4] 2> 2628168 T4918 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438499529864249344)]} 0 2
[junit4:junit4] 2> 2628173 T4936 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438499529864249344)]} 0 2
[junit4:junit4] 2> 2628174 T4904 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[1 (1438499529864249344)]} 0 14
[junit4:junit4] 2> 2628182 T4886 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[2 (1438499529884172288)]} 0 2
[junit4:junit4] 2> 2628192 T4902 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:51176/uu_qpg/tw/collection1/]
[junit4:junit4] 2> 2628193 T4902 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51176/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:true
[junit4:junit4] 2> 2628194 T4902 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51180/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:false
[junit4:junit4] 2> 2628195 T4902 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51187/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:false
[junit4:junit4] 2> 2628204 T4955 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[2 (1438499529900949504)]} 0 2
[junit4:junit4] 2> 2628204 T4938 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[2 (1438499529900949504)]} 0 2
[junit4:junit4] 2> 2628207 T4902 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51180/uu_qpg/tw/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[2 (1438499529900949504)]} 0 16
[junit4:junit4] 2> 2628208 T4937 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[2]} 0 21
[junit4:junit4] 2> 2628214 T4881 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={wt=javabin&version=2} {add=[3 (1438499529917726720)]} 0 2
[junit4:junit4] 2> 2628236 T4901 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:51176/uu_qpg/tw/collection1/]
[junit4:junit4] 2> 2628237 T4901 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51176/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:true
[junit4:junit4] 2> 2628237 T4901 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51180/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:false
[junit4:junit4] 2> 2628238 T4901 C1267 P51173 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:51187/uu_qpg/tw/collection1/ against:[http://127.0.0.1:51176/uu_qpg/tw/collection1/] result:false
[junit4:junit4] 2> 2628246 T4956 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[3 (1438499529944989696)]} 0 2
[junit4:junit4] 2> 2628246 T4939 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51173/uu_qpg/tw/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[3 (1438499529944989696)]} 0 2
[junit4:junit4] 2> 2628248 T4901 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={distrib.from=http://127.0.0.1:51176/uu_qpg/tw/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[3 (1438499529944989696)]} 0 12
[junit4:junit4] 2> 2628249 T4919 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={test.distrib.skip.servers=http://127.0.0.1:51176/uu_qpg/tw/collection1/&test.distrib.skip.servers=http://127.0.0.1:51180/uu_qpg/tw/collection1/&wt=javabin&version=2} {add=[3]} 0 25
[junit4:junit4] 2> 2628253 T4882 C1271 P51169 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2628359 T4882 C1271 P51169 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@e12f436; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 2628363 T4882 C1271 P51169 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 2628387 T4882 C1271 P51169 oass.SolrIndexSearcher.<init> Opening Searcher@33579950 main
[junit4:junit4] 2> 2628389 T4882 C1271 P51169 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2628392 T4894 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33579950 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 2628393 T4882 C1271 P51169 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 140
[junit4:junit4] 2> 2628398 T4905 C1267 P51173 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51173/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51176/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51180/uu_qpg/tw/collection1/, StdNode: http://127.0.0.1:51187/uu_qpg/tw/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 2628403 T4935 C1269 P51180 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2628404 T4958 C1270 P51187 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2628404 T4906 C1267 P51173 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2628404 T4920 C1268 P51176 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2628546 T4920 C1268 P51176 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@195c7c27; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 2628547 T4920 C1268 P51176 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 2628546 T4958 C1270 P51187 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@609317ca; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 2628549 T4958 C1270 P51187 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 2628561 T4920 C1268 P51176 oass.SolrIndexSearcher.<init> Opening Searcher@64a5f921 main
[junit4:junit4] 2> 2628562 T4920 C1268 P51176 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2628564 T4928 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64a5f921 main{StandardDirectoryReader(segments_4:3:nrt _0(5.0):C2)}
[junit4:junit4] 2> 2628565 T4920 C1268 P51176 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 161
[junit4:junit4] 2> 2628590 T4958 C1270 P51187 oass.SolrIndexSearcher.<init> Opening Searcher@1766fc90 main
[junit4:junit4] 2> 2628603 T4958 C1270 P51187 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2628604 T4966 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1766fc90 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 2628608 T4958 C1270 P51187 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 204
[junit4:junit4] 2> 2628629 T4935 C1269 P51180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@236911a1; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 2628630 T4935 C1269 P51180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 2628645 T4935 C1269 P51180 oass.SolrIndexSearcher.<init> Opening Searcher@249ba853 main
[junit4:junit4] 2> 2628646 T4935 C1269 P51180 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2628648 T4947 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@249ba853 main{StandardDirectoryReader(segments_5:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 2628658 T4935 C1269 P51180 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 255
[junit4:junit4] 2> 2628663 T4906 C1267 P51173 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371860022220/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61dfdac8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 2628665 T4906 C1267 P51173 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 2628680 T4906 C1267 P51173 oass.SolrIndexSearcher.<init> Opening Searcher@2c07b893 main
[junit4:junit4] 2> 2628681 T4906 C1267 P51173 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2628683 T4912 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c07b893 main{StandardDirectoryReader(segments_6:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 2628684 T4906 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 281
[junit4:junit4] 2> 2628686 T4905 C1267 P51173 oasup.LogUpdateProcessor.finish [collection1] webapp=/uu_qpg/tw path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 288
[junit4:junit4] 2> 2628688 T4868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 2628690 T4868 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2628693 T4868 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 2628702 T4904 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 2628708 T4922 C1268 P51176 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 2628723 T4936 C1269 P51180 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 2628728 T4953 C1270 P51187 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 2628729 T4868 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2628762 T4938 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 2628763 T4938 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2628805 T4902 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 2628806 T4902 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628811 T4902 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw START replicas=[http://127.0.0.1:51176/uu_qpg/tw/collection1/, http://127.0.0.1:51180/uu_qpg/tw/collection1/, http://127.0.0.1:51187/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2628822 T4917 C1268 P51176 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2628824 T4937 C1269 P51180 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2628832 T4954 C1270 P51187 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2628833 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Received 3 versions from 127.0.0.1:51176/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628834 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Our versions are newer. ourLowThreshold=1438499529749954560 otherHigh=1438499529864249344
[junit4:junit4] 2> 2628835 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Received 5 versions from 127.0.0.1:51180/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628835 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Our versions are newer. ourLowThreshold=1438499529749954560 otherHigh=1438499529900949504
[junit4:junit4] 2> 2628836 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Received 5 versions from 127.0.0.1:51187/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628836 T4902 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw Our versions are newer. ourLowThreshold=1438499529749954560 otherHigh=1438499529900949504
[junit4:junit4] 2> 2628837 T4902 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51173/uu_qpg/tw DONE. sync succeeded
[junit4:junit4] 2> 2628837 T4902 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2628838 T4902 oasc.SyncStrategy.syncToMe http://127.0.0.1:51173/uu_qpg/tw/collection1/: try and ask http://127.0.0.1:51176/uu_qpg/tw/collection1/ to sync
[junit4:junit4] 2> 2628839 T4902 oasc.SyncStrategy.syncToMe http://127.0.0.1:51173/uu_qpg/tw/collection1/: try and ask http://127.0.0.1:51180/uu_qpg/tw/collection1/ to sync
[junit4:junit4] 2> 2628840 T4902 oasc.SyncStrategy.syncToMe http://127.0.0.1:51173/uu_qpg/tw/collection1/: try and ask http://127.0.0.1:51187/uu_qpg/tw/collection1/ to sync
[junit4:junit4] 2> 2628845 T4918 C1268 P51176 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51176/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2628848 T4939 C1269 P51180 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51180/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2628850 T4955 C1270 P51187 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51187/uu_qpg/tw START replicas=[http://127.0.0.1:51173/uu_qpg/tw/collection1/] nUpdates=100
[junit4:junit4] 2> 2628853 T4901 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2628857 T4901 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2628857 T4918 C1268 P51176 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51176/uu_qpg/tw Received 5 versions from 127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628859 T4939 C1269 P51180 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51180/uu_qpg/tw Received 5 versions from 127.0.0.1:51173/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628861 T4939 C1269 P51180 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51180/uu_qpg/tw Our versions are newer. ourLowThreshold=1438499529749954560 otherHigh=1438499529900949504
[junit4:junit4] 2> 2628861 T4939 C1269 P51180 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51180/uu_qpg/tw DONE. sync succeeded
[junit4:junit4] 2> 2628862 T4939 C1269 P51180 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={sync=http://127.0.0.1:51173/uu_qpg/tw/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=15
[junit4:junit4] 2> 2628860 T4918 C1268 P51176 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:51176/uu_qpg/tw Requesting updates from 127.0.0.1:51173/uu_qpg/tw/collection1/n=2 versions=[1438499529944989696, 1438499529900949504]
[junit4:junit4] 2> 2628864 T4902 oasc.SyncStrategy.syncToMe http://127.0.0.1:51173/uu_qpg/tw/collection1/: sync completed with http://127.0.0.1:51180/uu_qpg/tw/collection1/
[junit4:junit4] 2> 2628865 T4903 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2628868 T4905 C1267 P51173 oasc.SolrCore.execute [collection1] webapp=/uu_qpg/tw path=/get params={distrib=false&getUpdates=1438499529944989696,1438499529900949504&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2628868 T4955 C1270 P51187 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:51187/uu_qpg/tw Received 5 versions from 127.0.0.1:51173/uu_qpg/tw/co
[...truncated too long message...]
830-127.0.0.1:51173_uu_qpg%2Ftw_collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89906218416930828-127.0.0.1:51187_uu_qpg%2Ftw_collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51180_uu_qpg%2Ftw",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51180/uu_qpg/tw"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89906218416930819-127.0.0.1:51169_uu_qpg%2Ftw_collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:51169_uu_qpg%2Ftw",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51169/uu_qpg/tw"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89906218416930828-127.0.0.1:51187_uu_qpg%2Ftw-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89906218416930819-127.0.0.1:51169_uu_qpg%2Ftw-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89906218416930830-127.0.0.1:51173_uu_qpg%2Ftw-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89906218416930826-127.0.0.1:51180_uu_qpg%2Ftw-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89906218416930819-127.0.0.1:51169_uu_qpg%2Ftw-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=74D793ECD44EB334 -Dtests.slow=true -Dtests.locale=en_AU -Dtests.timezone=Atlantic/Cape_Verde -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 126s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:51173/uu_qpg/tw/collection1lastClient and got 267 from http://127.0.0.1:51180/uu_qpg/tw/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([74D793ECD44EB334:F5311DF4A311D308]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2720567 T4868 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 126006 T4867 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 2721316 T4962 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2721316 T4962 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2721317 T4962 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=NestedPulsing), text=PostingsFormat(name=Memory doPackFST= true), _version_=PostingsFormat(name=NestedPulsing), rnd_b=PostingsFormat(name=Asserting), intDefault=PostingsFormat(name=NestedPulsing), timestamp=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=NestedPulsing), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=Lucene41(blocksize=128), other_tl1=PostingsFormat(name=NestedPulsing), multiDefault=Lucene41(blocksize=128), a_si=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=DefaultSimilarity, locale=en_AU, timezone=Atlantic/Cape_Verde
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=3,free=227106128,total=289406976
[junit4:junit4] 2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, SolrIndexSplitterTest, TestPHPSerializedResponseWriter, ChaosMonkeySafeLeaderTest, SpatialFilterTest, TestSchemaNameResource, RegexBoostProcessorTest, TestBinaryResponseWriter, TestElisionMultitermQuery, NoCacheHeaderTest, TestDocSet, TestFoldingMultitermQuery, ZkControllerTest, TermVectorComponentTest, UniqFieldsUpdateProcessorFactoryTest, PeerSyncTest, DistributedQueryElevationComponentTest, BasicFunctionalityTest, ZkCLITest, OverseerTest, DocValuesMultiTest, TestQueryUtils, FileBasedSpellCheckerTest, TestFieldTypeResource, TestQuerySenderListener, TermVectorComponentDistributedTest, HighlighterConfigTest, SampleTest, ReturnFieldsTest, TestRandomDVFaceting, TestMergePolicyConfig, TestReload, StandardRequestHandlerTest, TestStressRecovery, IndexReaderFactoryTest, TestAtomicUpdateErrorCases, SolrIndexConfigTest, CopyFieldTest, SuggesterTest, QueryResultKeyTest, TestFastOutputStream, TestManagedSchema, TestUtils, TestShardHandlerFactory, StatsComponentTest, TestPropInjectDefaults, PrimUtilsTest, ZkSolrClientTest, TestCodecSupport, SolrCoreCheckLockOnStartupTest, WordBreakSolrSpellCheckerTest, TestFiltering, TestBM25SimilarityFactory, DocumentAnalysisRequestHandlerTest, TestLFUCache, TestQuerySenderNoQuery, CoreContainerCoreInitFailuresTest, IndexBasedSpellCheckerTest, TestSchemaSimilarityResource, FastVectorHighlighterTest, TestFieldTypeCollectionResource, SignatureUpdateProcessorFactoryTest, TestSolrQueryParserResource, TestPerFieldSimilarity, DefaultValueUpdateProcessorTest, UpdateParamsTest, TestLazyCores, TestStressVersions, AliasIntegrationTest, MinimalSchemaTest, SliceStateUpdateTest, DirectUpdateHandlerTest, SolrInfoMBeanTest, LeaderElectionIntegrationTest, ClusterStateUpdateTest, TestSolr4Spatial, TestDocumentBuilder, DOMUtilTest, CollectionsAPIDistributedZkTest, TestLMJelinekMercerSimilarityFactory, QueryElevationComponentTest, TestXIncludeConfig, TestValueSourceCache, SimpleFacetsTest, TestRandomFaceting, TestDynamicFieldCollectionResource, MoreLikeThisHandlerTest, TestPhraseSuggestions, TestFastLRUCache, DirectUpdateHandlerOptimizeTest, TestUpdate, TestSolrQueryParser, TestDFRSimilarityFactory, TestZkChroot, SuggesterWFSTTest, OutputWriterTest, TestSurroundQueryParser, BasicDistributedZkTest, CurrencyFieldOpenExchangeTest, PingRequestHandlerTest, DocValuesTest, QueryEqualityTest, TestLMDirichletSimilarityFactory, TestSolrXml, CSVRequestHandlerTest, TestCoreDiscovery, BasicZkTest, MultiTermTest, TestCSVLoader, CircularListTest, IndexSchemaRuntimeFieldTest, TestPartialUpdateDeduplication, SolrRequestParserTest, ChaosMonkeyNothingIsSafeTest, NotRequiredUniqueKeyTest, TestHashPartitioner, TestLRUCache, TimeZoneUtilsTest, TestFuzzyAnalyzedSuggestions, TestRealTimeGet, RecoveryZkTest, TestMultiCoreConfBootstrap, TestSolrQueryParserDefaultOperatorResource, TestGroupingSearch, OpenExchangeRatesOrgProviderTest, SolrPluginUtilsTest, DistanceFunctionTest, DistributedTermsComponentTest, DirectSolrConnectionTest, TestJmxMonitoredMap, TestSolrDeletionPolicy1, TestStressLucene, SearchHandlerTest, TestExtendedDismaxParser, TestPostingsSolrHighlighter, EchoParamsTest, TestReplicationHandler, TestMaxScoreQueryParser, BadComponentTest, StatelessScriptUpdateProcessorFactoryTest, TestSort, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed in 127.08s, 1 test, 1 failure <<< FAILURES!
[...truncated 455 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1243: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:887: There were test failures: 297 suites, 1252 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 113 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure