You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/11/24 14:31:46 UTC

[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/
Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC

All tests passed

Build Log:
[...truncated 1126 lines...]
   [junit4] JVM J0: stdout was not empty, see: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.sysout
   [junit4] >>> JVM J0: stdout (verbatim) ----
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  SIGSEGV (0xb) at pc=0xf6fc55f1, pid=11654, tid=4098882368
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0-b116) (build 1.8.0-ea-b116)
   [junit4] # Java VM: Java HotSpot(TM) Client VM (25.0-b58 mixed mode linux-x86 )
   [junit4] # Problematic frame:
   [junit4] # V  [libjvm.so+0x4475f1]  ObjArrayKlass::oop_oop_iterate_nv(oopDesc*, ParScanWithoutBarrierClosure*)+0x61
   [junit4] #
   [junit4] # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/J0/hs_err_pid11654.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.sun.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J0: EOF ----

[...truncated 454 lines...]
   [junit4] ERROR: JVM J0 ended with an exception, command line: /var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/jre/bin/java -client -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/heapdumps -Dtests.prefix=tests -Dtests.seed=3861E4A6DB805158 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp -Dclover.db.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.0.13.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/test:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.events @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.suites
   [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:744)

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.

Total time: 14 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8353 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8353/
Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Could not find the new collection - 404 : http://127.0.0.1:55577/onenodecollectioncore

Stack Trace:
java.lang.AssertionError: Could not find the new collection - 404 : http://127.0.0.1:55577/onenodecollectioncore
	at __randomizedtesting.SeedInfo.seed([9EA504F3F84AB5CD:1F438AEB8F15D5F1]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1797)
	at org.apache.solr.cloud.BasicDistributedZk2Test.addAndQueryDocs(BasicDistributedZk2Test.java:206)
	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:195)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10506 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
   [junit4]   2> 685429 T2147 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 685432 T2147 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1385321955857
   [junit4]   2> 685432 T2147 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 685433 T2148 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 685533 T2147 oasc.ZkTestServer.run start zk server on port:48926
   [junit4]   2> 685534 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 685592 T2154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5262eb name:ZooKeeperConnection Watcher:127.0.0.1:48926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685592 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 685593 T2147 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 685597 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 685598 T2156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e3ba0 name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685598 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 685599 T2147 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 685602 T2147 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 685605 T2147 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 685606 T2147 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 685608 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 685608 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 685611 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 685611 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 685613 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685614 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 685615 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 685616 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 685617 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 685618 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 685619 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 685620 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 685622 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 685622 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 685624 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 685624 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 685626 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685626 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 685628 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 685629 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 685630 T2147 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 685631 T2147 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 685700 T2147 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 685703 T2147 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43702
   [junit4]   2> 685703 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 685704 T2147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 685704 T2147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059
   [junit4]   2> 685704 T2147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/'
   [junit4]   2> 685718 T2147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/solr.xml
   [junit4]   2> 685726 T2147 oasc.CoreContainer.<init> New CoreContainer 24461235
   [junit4]   2> 685727 T2147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/]
   [junit4]   2> 685727 T2147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 685728 T2147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 685728 T2147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 685728 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 685729 T2147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 685729 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 685729 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 685730 T2147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 685730 T2147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 685733 T2147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 685733 T2147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 685734 T2147 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 685734 T2147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48926/solr
   [junit4]   2> 685735 T2147 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 685735 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 685737 T2167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f7ede7 name:ZooKeeperConnection Watcher:127.0.0.1:48926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685737 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 685739 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 685741 T2169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f765a name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 685741 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 685742 T2147 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 685744 T2147 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 685747 T2147 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 685748 T2147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43702_
   [junit4]   2> 685749 T2147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43702_
   [junit4]   2> 685751 T2147 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 685755 T2147 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 685756 T2147 oasc.Overseer.start Overseer (id=90788459699240963-127.0.0.1:43702_-n_0000000000) starting
   [junit4]   2> 685758 T2147 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 685760 T2171 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 685761 T2147 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 685764 T2147 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 685765 T2147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 685767 T2170 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 685768 T2172 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 685768 T2172 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 685769 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 685769 T2172 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 685770 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 685770 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:43702",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:43702_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 685770 T2170 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 685770 T2170 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 685772 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 685773 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 686769 T2172 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 686770 T2172 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/collection1
   [junit4]   2> 686770 T2172 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 686771 T2172 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 686771 T2172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 686772 T2172 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/collection1/'
   [junit4]   2> 686773 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/collection1/lib/classes/' to classloader
   [junit4]   2> 686773 T2172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/collection1/lib/README' to classloader
   [junit4]   2> 686794 T2172 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 686803 T2172 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 686804 T2172 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 686815 T2172 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 686922 T2172 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 686922 T2172 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 686923 T2172 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 686927 T2172 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 686929 T2172 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 686941 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 686943 T2172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 686945 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 686946 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 686946 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 686946 T2172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 686947 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 686947 T2172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 686947 T2172 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 686948 T2172 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385321956059/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/control/data/
   [junit4]   2> 686948 T2172 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7913d
   [junit4]   2> 686948 T2172 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/control/data
   [junit4]   2> 686949 T2172 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/control/data/index/
   [junit4]   2> 686949 T2172 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 686949 T2172 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/control/data/index
   [junit4]   2> 686949 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=49, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.23780290728136252]
   [junit4]   2> 686950 T2172 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@77b57d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b5bb9f),segFN=segments_1,generation=1}
   [junit4]   2> 686950 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 686952 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 686953 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 686953 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 686953 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 686953 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 686953 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 686954 T2172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 686954 T2172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 686954 T2172 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 686955 T2172 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 686955 T2172 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 686955 T2172 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 686956 T2172 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 686956 T2172 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 686956 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 686957 T2172 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 686958 T2172 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 686960 T2172 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 686960 T2172 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 686961 T2172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=0.3310546875, floorSegmentMB=1.744140625, forceMergeDeletesPctAllowed=0.3255189490863808, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 686961 T2172 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@77b57d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b5bb9f),segFN=segments_1,generation=1}
   [junit4]   2> 686961 T2172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 686962 T2172 oass.SolrIndexSearcher.<init> Opening Searcher@1373bbe main
   [junit4]   2> 686964 T2173 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1373bbe main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 686965 T2172 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 686965 T2172 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43702 collection:control_collection shard:shard1
   [junit4]   2> 686966 T2172 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 686971 T2172 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 686972 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 686973 T2172 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 686973 T2172 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C7726 name=collection1 org.apache.solr.core.SolrCore@c2fc16 url=http://127.0.0.1:43702/collection1 node=127.0.0.1:43702_ C7726_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:43702, core=collection1, node_name=127.0.0.1:43702_}
   [junit4]   2> 686973 T2172 C7726 P43702 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43702/collection1/
   [junit4]   2> 686973 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 686973 T2172 C7726 P43702 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 686974 T2172 C7726 P43702 oasc.SyncStrategy.syncToMe http://127.0.0.1:43702/collection1/ has no replicas
   [junit4]   2> 686974 T2172 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43702/collection1/ shard1
   [junit4]   2> 686974 T2172 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 686975 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 686979 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 686983 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 687086 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687129 T2172 oasc.ZkController.register We are http://127.0.0.1:43702/collection1/ and leader is http://127.0.0.1:43702/collection1/
   [junit4]   2> 687130 T2172 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43702
   [junit4]   2> 687130 T2172 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 687130 T2172 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 687131 T2172 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 687132 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 687132 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 687133 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 687133 T2172 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 687134 T2147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 687135 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 687136 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 687137 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 687138 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:43702",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:43702_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 687139 T2176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15caa6b name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 687139 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 687141 T2147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 687142 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 687145 T2147 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 687225 T2147 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 687227 T2147 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49844
   [junit4]   2> 687228 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 687228 T2147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 687229 T2147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571
   [junit4]   2> 687229 T2147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/'
   [junit4]   2> 687242 T2147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/solr.xml
   [junit4]   2> 687244 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687244 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 687251 T2147 oasc.CoreContainer.<init> New CoreContainer 8776383
   [junit4]   2> 687252 T2147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/]
   [junit4]   2> 687252 T2147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 687253 T2147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 687253 T2147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 687253 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 687254 T2147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 687254 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 687254 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 687255 T2147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 687255 T2147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 687258 T2147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 687258 T2147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 687259 T2147 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 687259 T2147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48926/solr
   [junit4]   2> 687260 T2147 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 687261 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 687264 T2187 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@96d4a5 name:ZooKeeperConnection Watcher:127.0.0.1:48926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 687264 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 687268 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 687270 T2189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bc51b3 name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 687270 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 687276 T2147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 688279 T2147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49844_
   [junit4]   2> 688280 T2147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49844_
   [junit4]   2> 688283 T2176 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 688283 T2169 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 688283 T2189 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 688286 T2190 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 688286 T2190 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 688287 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 688287 T2190 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 688287 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 688288 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 688289 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 688290 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49844",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49844_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 688290 T2170 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 688290 T2170 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 688293 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 688396 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 688396 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 688396 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689288 T2190 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 689288 T2190 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/collection1
   [junit4]   2> 689288 T2190 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 689289 T2190 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 689289 T2190 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 689290 T2190 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/collection1/'
   [junit4]   2> 689291 T2190 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/collection1/lib/classes/' to classloader
   [junit4]   2> 689291 T2190 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/collection1/lib/README' to classloader
   [junit4]   2> 689313 T2190 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 689322 T2190 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 689323 T2190 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 689329 T2190 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 689440 T2190 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 689441 T2190 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 689442 T2190 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 689446 T2190 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 689447 T2190 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 689456 T2190 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 689458 T2190 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 689460 T2190 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 689460 T2190 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 689461 T2190 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 689461 T2190 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 689462 T2190 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 689462 T2190 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 689462 T2190 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 689462 T2190 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385321957571/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty1/
   [junit4]   2> 689463 T2190 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7913d
   [junit4]   2> 689463 T2190 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty1
   [junit4]   2> 689464 T2190 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty1/index/
   [junit4]   2> 689464 T2190 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 689464 T2190 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty1/index
   [junit4]   2> 689464 T2190 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=49, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.23780290728136252]
   [junit4]   2> 689465 T2190 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@d3cbcb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14d25c1),segFN=segments_1,generation=1}
   [junit4]   2> 689465 T2190 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 689468 T2190 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 689468 T2190 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 689468 T2190 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 689469 T2190 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 689469 T2190 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 689469 T2190 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 689469 T2190 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 689470 T2190 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 689470 T2190 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 689470 T2190 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 689471 T2190 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 689471 T2190 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 689472 T2190 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 689472 T2190 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 689472 T2190 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 689473 T2190 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 689475 T2190 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 689476 T2190 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 689476 T2190 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 689477 T2190 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=0.3310546875, floorSegmentMB=1.744140625, forceMergeDeletesPctAllowed=0.3255189490863808, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 689477 T2190 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@d3cbcb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14d25c1),segFN=segments_1,generation=1}
   [junit4]   2> 689478 T2190 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 689478 T2190 oass.SolrIndexSearcher.<init> Opening Searcher@1d8b84e main
   [junit4]   2> 689482 T2191 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d8b84e main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 689484 T2190 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 689484 T2190 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49844 collection:collection1 shard:shard2
   [junit4]   2> 689485 T2190 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 689490 T2190 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 689491 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689491 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689491 T2190 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 689492 T2190 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 689491 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C7727 name=collection1 org.apache.solr.core.SolrCore@48c032 url=http://127.0.0.1:49844/collection1 node=127.0.0.1:49844_ C7727_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:49844, core=collection1, node_name=127.0.0.1:49844_}
   [junit4]   2> 689492 T2190 C7727 P49844 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49844/collection1/
   [junit4]   2> 689492 T2190 C7727 P49844 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 689493 T2190 C7727 P49844 oasc.SyncStrategy.syncToMe http://127.0.0.1:49844/collection1/ has no replicas
   [junit4]   2> 689493 T2190 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49844/collection1/ shard2
   [junit4]   2> 689493 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 689493 T2190 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 689496 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689498 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689500 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689603 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689603 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689603 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689649 T2190 oasc.ZkController.register We are http://127.0.0.1:49844/collection1/ and leader is http://127.0.0.1:49844/collection1/
   [junit4]   2> 689649 T2190 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49844
   [junit4]   2> 689649 T2190 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 689649 T2190 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 689650 T2190 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 689651 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689652 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689652 T2190 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 689652 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689653 T2147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 689653 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 689653 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 689654 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49844",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49844_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 689656 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 689724 T2147 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 689726 T2147 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55577
   [junit4]   2> 689727 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 689728 T2147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 689728 T2147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080
   [junit4]   2> 689728 T2147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/'
   [junit4]   2> 689742 T2147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/solr.xml
   [junit4]   2> 689750 T2147 oasc.CoreContainer.<init> New CoreContainer 15298170
   [junit4]   2> 689751 T2147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/]
   [junit4]   2> 689751 T2147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 689752 T2147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 689752 T2147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 689752 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 689753 T2147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 689753 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 689753 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 689753 T2147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 689754 T2147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 689757 T2147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 689757 T2147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 689758 T2147 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 689758 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689758 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689758 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 689758 T2147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48926/solr
   [junit4]   2> 689760 T2147 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 689761 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 689763 T2203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15496b1 name:ZooKeeperConnection Watcher:127.0.0.1:48926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689763 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 689765 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 689766 T2205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e17aea name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 689767 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 689769 T2147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 690772 T2147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55577_
   [junit4]   2> 690773 T2147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55577_
   [junit4]   2> 690777 T2176 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 690777 T2205 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 690777 T2189 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 690778 T2169 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 690781 T2206 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 690781 T2206 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 690782 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 690783 T2206 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 690783 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 690783 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 690784 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 690785 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:55577",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55577_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 690785 T2170 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 690785 T2170 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 690787 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 690889 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 690889 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 690889 T2205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 690889 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 691783 T2206 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 691783 T2206 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/collection1
   [junit4]   2> 691784 T2206 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 691784 T2206 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 691785 T2206 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 691786 T2206 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/collection1/'
   [junit4]   2> 691786 T2206 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/collection1/lib/classes/' to classloader
   [junit4]   2> 691787 T2206 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/collection1/lib/README' to classloader
   [junit4]   2> 691815 T2206 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 691825 T2206 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 691826 T2206 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 691832 T2206 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 691952 T2206 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 691953 T2206 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 691953 T2206 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 691957 T2206 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 691959 T2206 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 691970 T2206 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 691973 T2206 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 691975 T2206 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 691976 T2206 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 691977 T2206 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 691977 T2206 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 691978 T2206 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 691978 T2206 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 691978 T2206 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 691979 T2206 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385321960080/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty2/
   [junit4]   2> 691979 T2206 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7913d
   [junit4]   2> 691980 T2206 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty2
   [junit4]   2> 691980 T2206 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty2/index/
   [junit4]   2> 691980 T2206 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 691980 T2206 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385321955857/jetty2/index
   [junit4]   2> 691981 T2206 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=49, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.23780290728136252]
   [junit4]   2> 691981 T2206 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@15a7c6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b3511),segFN=segments_1,generation=1}
   [junit4]   2> 691982 T2206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 691984 T2206 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 691984 T2206 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 691985 T2206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 691985 T2206 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 691985 T2206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 691985 T2206 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 691986 T2206 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 691986 T2206 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 691986 T2206 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 691987 T2206 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 691988 T2206 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 691988 T2206 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 691988 T2206 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 691989 T2206 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 691989 T2206 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 691990 T2206 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 691992 T2206 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 691993 T2206 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 691993 T2206 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 691995 T2206 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=0.3310546875, floorSegmentMB=1.744140625, forceMergeDeletesPctAllowed=0.3255189490863808, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 691995 T2206 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@15a7c6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1b3511),segFN=segments_1,generation=1}
   [junit4]   2> 691996 T2206 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 691996 T2206 oass.SolrIndexSearcher.<init> Opening Searcher@89396f main
   [junit4]   2> 692001 T2207 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@89396f main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 692003 T2206 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 692003 T2206 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55577 collection:collection1 shard:shard1
   [junit4]   2> 692004 T2206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 692010 T2206 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 692011 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692012 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692012 T2206 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 692012 T2206 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 692012 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> ASYNC  NEW_CORE C7728 name=collection1 org.apache.solr.core.SolrCore@1425c27 url=http://127.0.0.1:55577/collection1 node=127.0.0.1:55577_ C7728_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:55577, core=collection1, node_name=127.0.0.1:55577_}
   [junit4]   2> 692013 T2206 C7728 P55577 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55577/collection1/
   [junit4]   2> 692013 T2206 C7728 P55577 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 692013 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 692013 T2206 C7728 P55577 oasc.SyncStrategy.syncToMe http://127.0.0.1:55577/collection1/ has no replicas
   [junit4]   2> 692014 T2206 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55577/collection1/ shard1
   [junit4]   2> 692014 T2206 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 692016 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692021 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692025 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692128 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692128 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692128 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692128 T2205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692172 T2206 oasc.ZkController.register We are http://127.0.0.1:55577/collection1/ and leader is http://127.0.0.1:55577/collection1/
   [junit4]   2> 692172 T2206 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55577
   [junit4]   2> 692172 T2206 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 692173 T2206 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 692173 T2206 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 692174 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692174 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692175 T2206 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 692175 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692176 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 692176 T2147 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 692176 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 692176 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:55577",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:55577_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 692179 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 692248 T2147 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 692250 T2147 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49656
   [junit4]   2> 692251 T2147 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 692251 T2147 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 692251 T2147 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602
   [junit4]   2> 692252 T2147 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/'
   [junit4]   2> 692266 T2147 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/solr.xml
   [junit4]   2> 692274 T2147 oasc.CoreContainer.<init> New CoreContainer 24501096
   [junit4]   2> 692275 T2147 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/]
   [junit4]   2> 692276 T2147 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 692276 T2147 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 692276 T2147 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 692277 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 692277 T2147 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 692277 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 692278 T2147 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 692278 T2147 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 692278 T2147 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 692281 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692282 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692282 T2205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692282 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 692281 T2147 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 692283 T2147 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 692283 T2147 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 692284 T2147 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48926/solr
   [junit4]   2> 692284 T2147 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 692285 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 692287 T2219 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d8158b name:ZooKeeperConnection Watcher:127.0.0.1:48926 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692288 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 692293 T2147 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 692295 T2221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f9520 name:ZooKeeperConnection Watcher:127.0.0.1:48926/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692295 T2147 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 692300 T2147 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 693302 T2147 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49656_
   [junit4]   2> 693304 T2147 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49656_
   [junit4]   2> 693308 T2205 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 693308 T2176 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 693308 T2169 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 693308 T2189 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 693308 T2221 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 693314 T2222 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 693315 T2222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 693316 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 693317 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 693317 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 693316 T2222 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 693319 T2170 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 693320 T2170 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49656",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49656_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 693320 T2170 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 693320 T2170 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 693323 T2169 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 693426 T2176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 693426 T2205 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 693426 T2189 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 693426 T2169 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 693426 T2221 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 694317 T2222 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 694318 T2222 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/collection1
   [junit4]   2> 694318 T2222 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 694319 T2222 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 694319 T2222 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 694320 T2222 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/collection1/'
   [junit4]   2> 694321 T2222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/collection1/lib/classes/' to classloader
   [junit4]   2> 694322 T2222 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385321962602/collection1/lib/README' to classloader
   [junit4]   2> 694351 T2222 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 694363 T2222 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 694364 T2222 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 694373 T2222 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 694496 T2222 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 694497 T2222 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 694498 T2222 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 694502 T2222 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 694504 T2222 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 694513 T2222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 694515 T2222 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 694518 T2222 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 694519 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 694519 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 694519 T2222 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 694520 T2222 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 694520 T2222

[...truncated too long message...]

:
   [junit4]   1>        {"id":"90788459699240963-127.0.0.1:43702_-n_0000000000"}
   [junit4]   1>    /solr/overseer_elect/election (5)
   [junit4]   1>     /solr/overseer_elect/election/90788459699240963-127.0.0.1:43702_-n_0000000000 (0)
   [junit4]   1>     /solr/overseer_elect/election/90788459699240970-127.0.0.1:49656_-n_0000000003 (0)
   [junit4]   1>     /solr/overseer_elect/election/90788459699240968-127.0.0.1:55577_-n_0000000002 (0)
   [junit4]   1>     /solr/overseer_elect/election/90788459699240966-127.0.0.1:49844_-n_0000000001 (0)
   [junit4]   1>     /solr/overseer_elect/election/90788459699240972-127.0.0.1:35775_-n_0000000004 (0)
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {
   [junit4]   1>         "onenodecollection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:49844",
   [junit4]   1>                   "core":"onenodecollectioncore",
   [junit4]   1>                   "roles":"none",
   [junit4]   1>                   "node_name":"127.0.0.1:49844_",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1"},
   [junit4]   1>         "collection1":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:55577",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:55577_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:35775",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:35775_"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:49844",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:49844_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:49656",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:49656_"}}}},
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1"},
   [junit4]   1>         "control_collection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:43702",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:43702_",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1"}}
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=9EA504F3F84AB5CD -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_SG -Dtests.timezone=Egypt -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 51.6s J1 | BasicDistributedZk2Test.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Could not find the new collection - 404 : http://127.0.0.1:55577/onenodecollectioncore
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9EA504F3F84AB5CD:1F438AEB8F15D5F1]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForNon403or404or503(AbstractFullDistribZkTestBase.java:1797)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.addAndQueryDocs(BasicDistributedZk2Test.java:206)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:195)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:90)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 737057 T2147 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 51637 T2146 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> 737202 T2221 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 738304 T2238 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene46: {_version_=PostingsFormat(name=SimpleText), multiDefault=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=SimpleText), range_facet_sl=PostingsFormat(name=Asserting), range_facet_si=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Asserting), range_facet_l=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=en_SG, timezone=Egypt
   [junit4]   2> NOTE: Linux 3.8.0-33-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=175490480,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestRandomFaceting, TestMaxScoreQueryParser, TestBinaryField, TestDynamicFieldResource, TestQueryTypes, TermVectorComponentDistributedTest, TestCodecSupport, FileUtilsTest, TestJmxIntegration, SortByFunctionTest, CollectionsAPIDistributedZkTest, CachingDirectoryFactoryTest, TestHashPartitioner, TestUtils, TestImplicitCoreProperties, TestUpdate, TestCollationKeyRangeQueries, NumericFieldsTest, TestFastOutputStream, TestLazyCores, TestPhraseSuggestions, TestManagedSchemaFieldResource, SpellingQueryConverterTest, TestCollationField, SolrIndexConfigTest, TestArbitraryIndexDir, BlockCacheTest, SpellCheckCollatorTest, BlockDirectoryTest, TestCollapseQParserPlugin, SpellCheckComponentTest, ChaosMonkeySafeLeaderTest, DistanceFunctionTest, TestQuerySenderListener, TestBinaryResponseWriter, TestSolr4Spatial, TestDFRSimilarityFactory, TestFieldTypeCollectionResource, PathHierarchyTokenizerFactoryTest, CurrencyFieldOpenExchangeTest, TestStressReorder, TestOmitPositions, TestSolrDeletionPolicy2, TestCSVLoader, SolrXmlInZkTest, TestQuerySenderNoQuery, HdfsDirectoryTest, MigrateRouteKeyTest, TestSchemaVersionResource, RequiredFieldsTest, TestAddFieldRealTimeGet, TestDistribDocBasedVersion, SolrCoreCheckLockOnStartupTest, DeleteReplicaTest, TestNumberUtils, StandardRequestHandlerTest, TestIndexSearcher, NoCacheHeaderTest, TestSweetSpotSimilarityFactory, IndexBasedSpellCheckerTest, TestPartialUpdateDeduplication, ShardRoutingCustomTest, TestFiltering, NotRequiredUniqueKeyTest, FieldAnalysisRequestHandlerTest, UniqFieldsUpdateProcessorFactoryTest, DirectUpdateHandlerTest, TestStressRecovery, MinimalSchemaTest, TestSolrCoreProperties, SolrRequestParserTest, InfoHandlerTest, TestWordDelimiterFilterFactory, SolrTestCaseJ4Test, BasicDistributedZk2Test]
   [junit4] Completed on J1 in 52.92s, 1 test, 1 failure <<< FAILURES!

[...truncated 566 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:492: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: There were test failures: 345 suites, 1505 tests, 1 failure, 35 ignored (6 assumptions)

Total time: 46 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (64bit/ibm-j9-jdk6) - Build # 8352 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8352/
Java: 64bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}

1 tests failed.
REGRESSION:  org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
	at __randomizedtesting.SeedInfo.seed([3EC9FA2C2A43E16D:65F901E429044211]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.checkNumDocs(SignatureUpdateProcessorFactoryTest.java:71)
	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection(SignatureUpdateProcessorFactoryTest.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:738)




Build Log:
[...truncated 9286 lines...]
   [junit4] Suite: org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest
   [junit4]   2> 22261 T23 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658
   [junit4]   2> 22262 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 22264 T23 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 22265 T23 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 22351 T23 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 22437 T23 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 22440 T23 oass.IndexSchema.readSchema Reading Solr Schema from schema12.xml
   [junit4]   2> 22460 T23 oass.IndexSchema.readSchema [null] Schema name=test
   [junit4]   2> 23043 T23 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 23046 T23 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 23049 T23 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 23104 T23 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 23110 T23 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 23116 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 23117 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 23118 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 23138 T23 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 23139 T23 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr
   [junit4]   2> 23140 T23 oasc.SolrResourceLoader.<init> new SolrResourceLoader for deduced Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/'
   [junit4]   2> 23245 T23 oasc.CoreContainer.<init> New CoreContainer 1998223130
   [junit4]   2> 23246 T23 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/]
   [junit4]   2> 23248 T23 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 23249 T23 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 23250 T23 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 23251 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 23252 T23 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 23252 T23 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 23253 T23 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 23254 T23 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 23255 T23 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 23269 T23 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 23269 T23 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 23270 T23 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 23279 T24 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1
   [junit4]   2> 23279 T24 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/'
   [junit4]   2> 23282 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 23283 T24 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 23378 T24 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 23471 T24 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 23473 T24 oass.IndexSchema.readSchema Reading Solr Schema from schema12.xml
   [junit4]   2> 23490 T24 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 24002 T24 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 24006 T24 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 24008 T24 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 24062 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 24067 T24 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 24073 T24 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 24073 T24 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/
   [junit4]   2> 24074 T24 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
   [junit4]   2> 24075 T24 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658
   [junit4]   2> 24076 T24 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index/
   [junit4]   2> 24076 T24 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index' doesn't exist. Creating new index...
   [junit4]   2> 24077 T24 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index
   [junit4]   2> 24077 T24 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=37, maxMergedSegmentMB=59.4072265625, floorSegmentMB=1.244140625, forceMergeDeletesPctAllowed=4.140046340240585, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 24079 T24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_1,generation=1}
   [junit4]   2> 24079 T24 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 24080 T24 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 24082 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 24083 T24 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 24084 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 24085 T24 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 24086 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 24086 T24 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 24087 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 24088 T24 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 24088 T24 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 24089 T24 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 24089 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 24090 T24 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 24090 T24 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 24091 T24 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 24091 T24 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 24092 T24 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 24092 T24 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 24093 T24 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 24093 T24 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 24094 T24 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 24094 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 24095 T24 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 24095 T24 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 24096 T24 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 24096 T24 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 24097 T24 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 24097 T24 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24098 T24 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24098 T24 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24099 T24 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24099 T24 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24100 T24 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24100 T24 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24101 T24 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 24101 T24 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 24102 T24 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 24103 T24 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 24105 T24 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 24107 T24 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 24108 T24 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 24108 T24 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=1.640625, floorSegmentMB=1.5068359375, forceMergeDeletesPctAllowed=28.683567529152683, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.11907100365917866
   [junit4]   2> 24109 T24 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_1,generation=1}
   [junit4]   2> 24110 T24 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 24111 T24 oass.SolrIndexSearcher.<init> Opening Searcher@7f7c7f7c main
   [junit4]   2> 24111 T24 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 24169 T24 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 24230 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f7c7f7c main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 24231 T24 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 24234 T23 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 24246 T23 oas.SolrTestCaseJ4.setUp ###Starting testNonIndexedSignatureField
   [junit4]   2> ASYNC  NEW_CORE C21 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 24257 T23 C21 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> ASYNC  NEW_CORE C22 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 24268 T23 C22 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 12
   [junit4]   2> 24285 T23 C22 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 24286 T23 C22 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_2,generation=2}
   [junit4]   2> 24287 T23 C22 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 24289 T23 C22 oass.SolrIndexSearcher.<init> Opening Searcher@75867586 main
   [junit4]   2> 24290 T23 C22 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 24290 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75867586 main{StandardDirectoryReader(segments_2:2:nrt)}
   [junit4]   2> 24295 T23 C22 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 11
   [junit4]   2> 24321 T23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 24342 T23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_3,generation=3}
   [junit4]   2> 24343 T23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 24361 T23 oass.SolrIndexSearcher.<init> Opening Searcher@21b221b2 main
   [junit4]   2> 24363 T23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 24363 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@21b221b2 main{StandardDirectoryReader(segments_3:4:nrt _0(4.7):C2)}
   [junit4]   2> 24364 T23 oas.SolrTestCaseJ4.tearDown ###Ending testNonIndexedSignatureField
   [junit4]   2> 24381 T23 oas.SolrTestCaseJ4.setUp ###Starting testFailNonIndexedSigWithOverwriteDupes
   [junit4]   2> 24944 T23 C22 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 24946 T23 C22 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 555
   [junit4]   2> 24994 T23 C22 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 24996 T23 C22 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_3,generation=3}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_4,generation=4}
   [junit4]   2> 24997 T23 C22 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
   [junit4]   2> ASYNC  NEW_CORE C23 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 25023 T23 C23 oass.SolrIndexSearcher.<init> Opening Searcher@6c7a6c7a main
   [junit4]   2> 25024 T23 C23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25024 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6c7a6c7a main{StandardDirectoryReader(segments_4:5:nrt)}
   [junit4]   2> 25026 T23 C23 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 32
   [junit4]   2> 25029 T23 oas.SolrTestCaseJ4.tearDown ###Ending testFailNonIndexedSigWithOverwriteDupes
   [junit4]   2> 25050 T23 oas.SolrTestCaseJ4.setUp ###Starting testDupeDetection
   [junit4]   2> 25060 T23 C23 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 25061 T23 C23 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 2
   [junit4]   2> 25072 T23 C23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25074 T23 C23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_4,generation=4}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_5,generation=5}
   [junit4]   2> 25075 T23 C23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
   [junit4]   2> 25076 T23 C23 oass.SolrIndexSearcher.<init> Opening Searcher@2a1b2a1b main
   [junit4]   2> 25077 T23 C23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25077 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a1b2a1b main{StandardDirectoryReader(segments_5:6:nrt)}
   [junit4]   2> 25079 T23 C23 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 7
   [junit4]   2> 25091 T23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25129 T23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_5,generation=5}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_6,generation=6}
   [junit4]   2> 25131 T23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
   [junit4]   2> 25132 T23 oass.SolrIndexSearcher.<init> Opening Searcher@111e111e main
   [junit4]   2> 25133 T23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25133 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@111e111e main{StandardDirectoryReader(segments_6:8:nrt _1(4.7):C2)}
   [junit4]   2> 25140 T23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25158 T23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_6,generation=6}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_7,generation=7}
   [junit4]   2> 25159 T23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
   [junit4]   2> 25162 T23 oass.SolrIndexSearcher.<init> Opening Searcher@20922092 main
   [junit4]   2> 25163 T23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25164 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20922092 main{StandardDirectoryReader(segments_7:11:nrt _1(4.7):C2/1:delGen=1 _2(4.7):C1)}
   [junit4]   2> 25165 T23 oas.SolrTestCaseJ4.tearDown ###Ending testDupeDetection
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SignatureUpdateProcessorFactoryTest -Dtests.method=testDupeDetection -Dtests.seed=3EC9FA2C2A43E16D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=pt -Dtests.timezone=Etc/GMT+8 -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.19s J1 | SignatureUpdateProcessorFactoryTest.testDupeDetection <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<1> but was:<2>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3EC9FA2C2A43E16D:65F901E429044211]:0)
   [junit4]    > 	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.checkNumDocs(SignatureUpdateProcessorFactoryTest.java:71)
   [junit4]    > 	at org.apache.solr.update.processor.SignatureUpdateProcessorFactoryTest.testDupeDetection(SignatureUpdateProcessorFactoryTest.java:119)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:738)
   [junit4]   2> 25237 T23 oas.SolrTestCaseJ4.setUp ###Starting testMultiThreaded
   [junit4]   2> ASYNC  NEW_CORE C24 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 25247 T23 C24 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 25248 T23 C24 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> 25259 T23 C24 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25261 T23 C24 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_7,generation=7}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_8,generation=8}
   [junit4]   2> 25262 T23 C24 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 8
   [junit4]   2> 25264 T23 C24 oass.SolrIndexSearcher.<init> Opening Searcher@47ac47ac main
   [junit4]   2> 25266 T23 C24 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25266 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@47ac47ac main{StandardDirectoryReader(segments_8:12:nrt)}
   [junit4]   2> 25268 T23 C24 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 9
   [junit4]   2> 25282 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 25399 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_8,generation=8}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_9,generation=9}
   [junit4]   2> 25400 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 9
   [junit4]   2> 25716 T34 oass.SolrIndexSearcher.<init> Opening Searcher@23662366 main
   [junit4]   2> 25717 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 25717 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23662366 main{StandardDirectoryReader(segments_9:26:nrt _5(4.7):C69/68:delGen=1 _6(4.7):C60/59:delGen=1)}
   [junit4]   2> 25718 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26009 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_9,generation=9}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_a,generation=10}
   [junit4]   2> 26010 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 10
   [junit4]   2> 26032 T35 oass.SolrIndexSearcher.<init> Opening Searcher@73bc73bc main
   [junit4]   2> 26033 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26033 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73bc73bc main{StandardDirectoryReader(segments_a:34:nrt _9(4.7):C3/1:delGen=1)}
   [junit4]   2> 26033 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26036 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_a,generation=10}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_b,generation=11}
   [junit4]   2> 26036 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 11
   [junit4]   2> 26043 T36 oass.SolrIndexSearcher.<init> Opening Searcher@57e857e8 main
   [junit4]   2> 26044 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26044 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@57e857e8 main{StandardDirectoryReader(segments_b:37:nrt _9(4.7):C3/2:delGen=1 _a(4.7):C1)}
   [junit4]   2> 26044 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26053 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_b,generation=11}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_c,generation=12}
   [junit4]   2> 26053 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 12
   [junit4]   2> 26064 T34 oass.SolrIndexSearcher.<init> Opening Searcher@12691269 main
   [junit4]   2> 26065 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26065 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12691269 main{StandardDirectoryReader(segments_c:41:nrt _b(4.7):C1)}
   [junit4]   2> 26086 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26088 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_c,generation=12}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_d,generation=13}
   [junit4]   2> 26089 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 13
   [junit4]   2> 26098 T35 oass.SolrIndexSearcher.<init> Opening Searcher@1ac01ac0 main
   [junit4]   2> 26099 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26099 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ac01ac0 main{StandardDirectoryReader(segments_d:45:nrt _c(4.7):C1)}
   [junit4]   2> 26100 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26106 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_d,generation=13}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_e,generation=14}
   [junit4]   2> 26107 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 14
   [junit4]   2> 26120 T36 oass.SolrIndexSearcher.<init> Opening Searcher@2c902c9 main
   [junit4]   2> 26122 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26122 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c902c9 main{StandardDirectoryReader(segments_e:49:nrt _d(4.7):C1)}
   [junit4]   2> 26122 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26125 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_e,generation=14}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_f,generation=15}
   [junit4]   2> 26125 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 15
   [junit4]   2> 26134 T34 oass.SolrIndexSearcher.<init> Opening Searcher@630c630c main
   [junit4]   2> 26135 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26135 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@630c630c main{StandardDirectoryReader(segments_f:53:nrt _e(4.7):C1)}
   [junit4]   2> 26135 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26141 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_f,generation=15}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_g,generation=16}
   [junit4]   2> 26142 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 16
   [junit4]   2> 26156 T35 oass.SolrIndexSearcher.<init> Opening Searcher@7e897e89 main
   [junit4]   2> 26158 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26158 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e897e89 main{StandardDirectoryReader(segments_g:57:nrt _f(4.7):C1)}
   [junit4]   2> 26158 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26160 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_g,generation=16}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_h,generation=17}
   [junit4]   2> 26161 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 17
   [junit4]   2> 26167 T36 oass.SolrIndexSearcher.<init> Opening Searcher@45e245e2 main
   [junit4]   2> 26168 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26168 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45e245e2 main{StandardDirectoryReader(segments_h:61:nrt _g(4.7):C1)}
   [junit4]   2> 26168 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26174 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_h,generation=17}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_i,generation=18}
   [junit4]   2> 26175 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 18
   [junit4]   2> 26186 T34 oass.SolrIndexSearcher.<init> Opening Searcher@4c524c52 main
   [junit4]   2> 26187 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26188 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26188 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c524c52 main{StandardDirectoryReader(segments_i:65:nrt _h(4.7):C1)}
   [junit4]   2> 26190 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_i,generation=18}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_j,generation=19}
   [junit4]   2> 26191 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 19
   [junit4]   2> 26192 T35 oass.SolrIndexSearcher.<init> Opening Searcher@7f277f27 main
   [junit4]   2> 26193 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26194 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26195 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f277f27 main{StandardDirectoryReader(segments_i:65:nrt _h(4.7):C1)}
   [junit4]   2> 26214 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_j,generation=19}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_k,generation=20}
   [junit4]   2> 26215 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 20
   [junit4]   2> 26230 T36 oass.SolrIndexSearcher.<init> Opening Searcher@d7b0d7b main
   [junit4]   2> 26231 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26231 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d7b0d7b main{StandardDirectoryReader(segments_k:71:nrt _h(4.7):C1 _j(4.7):C1)}
   [junit4]   2> 26232 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26234 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_k,generation=20}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_l,generation=21}
   [junit4]   2> 26235 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 21
   [junit4]   2> 26250 T34 oass.SolrIndexSearcher.<init> Opening Searcher@416a416a main
   [junit4]   2> 26251 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26252 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@416a416a main{StandardDirectoryReader(segments_l:75:nrt _h(4.7):C1 _k(4.7):C1)}
   [junit4]   2> 26252 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26255 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_l,generation=21}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_m,generation=22}
   [junit4]   2> 26256 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 22
   [junit4]   2> 26268 T35 oass.SolrIndexSearcher.<init> Opening Searcher@28382838 main
   [junit4]   2> 26269 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26269 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28382838 main{StandardDirectoryReader(segments_m:79:nrt _k(4.7):C1 _l(4.7):C1)}
   [junit4]   2> 26270 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26276 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_m,generation=22}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_n,generation=23}
   [junit4]   2> 26277 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 23
   [junit4]   2> 26292 T36 oass.SolrIndexSearcher.<init> Opening Searcher@72af72af main
   [junit4]   2> 26293 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26293 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@72af72af main{StandardDirectoryReader(segments_n:83:nrt _k(4.7):C1 _m(4.7):C1)}
   [junit4]   2> 26294 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26296 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_n,generation=23}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_o,generation=24}
   [junit4]   2> 26296 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 24
   [junit4]   2> 26304 T34 oass.SolrIndexSearcher.<init> Opening Searcher@702b702b main
   [junit4]   2> 26305 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26305 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@702b702b main{StandardDirectoryReader(segments_o:87:nrt _m(4.7):C1 _n(4.7):C1)}
   [junit4]   2> 26305 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26312 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_o,generation=24}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_p,generation=25}
   [junit4]   2> 26313 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 25
   [junit4]   2> 26328 T35 oass.SolrIndexSearcher.<init> Opening Searcher@f900f90 main
   [junit4]   2> 26329 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26329 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f900f90 main{StandardDirectoryReader(segments_p:93:nrt _o(4.7):C1)}
   [junit4]   2> 26330 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26332 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_p,generation=25}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_q,generation=26}
   [junit4]   2> 26333 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 26
   [junit4]   2> 26341 T36 oass.SolrIndexSearcher.<init> Opening Searcher@39b939b9 main
   [junit4]   2> 26342 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26342 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39b939b9 main{StandardDirectoryReader(segments_q:97:nrt _p(4.7):C1)}
   [junit4]   2> 26350 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26354 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_q,generation=26}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_r,generation=27}
   [junit4]   2> 26355 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 27
   [junit4]   2> 26362 T34 oass.SolrIndexSearcher.<init> Opening Searcher@65d865d8 main
   [junit4]   2> 26363 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26364 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26367 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65d865d8 main{StandardDirectoryReader(segments_r:101:nrt _q(4.7):C1)}
   [junit4]   2> 26369 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_r,generation=27}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_s,generation=28}
   [junit4]   2> 26370 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 28
   [junit4]   2> 26377 T35 oass.SolrIndexSearcher.<init> Opening Searcher@20f020f main
   [junit4]   2> 26378 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26378 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20f020f main{StandardDirectoryReader(segments_s:103:nrt _q(4.7):C1 _r(4.7):C1)}
   [junit4]   2> 26379 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26384 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_s,generation=28}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_t,generation=29}
   [junit4]   2> 26385 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 29
   [junit4]   2> 26395 T36 oass.SolrIndexSearcher.<init> Opening Searcher@6e086e08 main
   [junit4]   2> 26396 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26396 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e086e08 main{StandardDirectoryReader(segments_t:107:nrt _q(4.7):C1 _s(4.7):C1)}
   [junit4]   2> 26397 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26399 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_t,generation=29}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_u,generation=30}
   [junit4]   2> 26400 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 30
   [junit4]   2> 26411 T34 oass.SolrIndexSearcher.<init> Opening Searcher@2dfe2dfe main
   [junit4]   2> 26412 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26413 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dfe2dfe main{StandardDirectoryReader(segments_u:111:nrt _t(4.7):C1)}
   [junit4]   2> 26413 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26415 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_u,generation=30}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_v,generation=31}
   [junit4]   2> 26416 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 31
   [junit4]   2> 26428 T35 oass.SolrIndexSearcher.<init> Opening Searcher@6a306a30 main
   [junit4]   2> 26430 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26430 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a306a30 main{StandardDirectoryReader(segments_v:115:nrt _u(4.7):C1)}
   [junit4]   2> 26430 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26433 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_v,generation=31}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_w,generation=32}
   [junit4]   2> 26434 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 32
   [junit4]   2> 26441 T36 oass.SolrIndexSearcher.<init> Opening Searcher@74c074c main
   [junit4]   2> 26441 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26442 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74c074c main{StandardDirectoryReader(segments_w:119:nrt _v(4.7):C1)}
   [junit4]   2> 26442 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26446 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_w,generation=32}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_x,generation=33}
   [junit4]   2> 26447 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 33
   [junit4]   2> 26454 T34 oass.SolrIndexSearcher.<init> Opening Searcher@75ba75ba main
   [junit4]   2> 26463 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26463 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75ba75ba main{StandardDirectoryReader(segments_x:123:nrt _w(4.7):C1)}
   [junit4]   2> 26464 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26466 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_x,generation=33}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_y,generation=34}
   [junit4]   2> 26467 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 34
   [junit4]   2> 26477 T35 oass.SolrIndexSearcher.<init> Opening Searcher@7ac57ac5 main
   [junit4]   2> 26479 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ac57ac5 main{StandardDirectoryReader(segments_y:127:nrt _x(4.7):C1)}
   [junit4]   2> 26480 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26480 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26482 T36 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_y,generation=34}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_z,generation=35}
   [junit4]   2> 26483 T36 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 35
   [junit4]   2> 26489 T36 oass.SolrIndexSearcher.<init> Opening Searcher@12e912e9 main
   [junit4]   2> 26490 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26491 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12e912e9 main{StandardDirectoryReader(segments_z:129:nrt _x(4.7):C1 _y(4.7):C1)}
   [junit4]   2> 26491 T34 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26497 T34 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_z,generation=35}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_10,generation=36}
   [junit4]   2> 26497 T34 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 36
   [junit4]   2> 26560 T34 oass.SolrIndexSearcher.<init> Opening Searcher@25642564 main
   [junit4]   2> 26561 T34 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26561 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@25642564 main{StandardDirectoryReader(segments_10:135:nrt _z(4.7):C1)}
   [junit4]   2> 26563 T35 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26564 T35 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_10,generation=36}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_11,generation=37}
   [junit4]   2> 26565 T35 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 37
   [junit4]   2> 26566 T35 oass.SolrIndexSearcher.<init> Opening Searcher@23b823b8 main
   [junit4]   2> 26568 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@23b823b8 main{StandardDirectoryReader(segments_10:135:nrt _z(4.7):C1)}
   [junit4]   2> 26568 T35 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26569 T36 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26570 T36 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 26571 T36 oass.SolrIndexSearcher.<init> Opening Searcher@73d073d main
   [junit4]   2> 26572 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73d073d main{StandardDirectoryReader(segments_10:135:nrt _z(4.7):C1)}
   [junit4]   2> 26573 T36 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> ASYNC  NEW_CORE C25 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 26582 T23 C25 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26583 T23 C25 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 26583 T23 C25 oass.SolrIndexSearcher.<init> Opening Searcher@73e473e4 main
   [junit4]   2> 26584 T23 C25 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26584 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73e473e4 main{StandardDirectoryReader(segments_10:135:nrt _z(4.7):C1)}
   [junit4]   2> 26586 T23 C25 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 4
   [junit4]   2> 26587 T23 oas.SolrTestCaseJ4.tearDown ###Ending testMultiThreaded
   [junit4]   2> 26605 T23 oas.SolrTestCaseJ4.setUp ###Starting testDupeAllFieldsDetection
   [junit4]   2> ASYNC  NEW_CORE C26 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 26616 T23 C26 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 26617 T23 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 2
   [junit4]   2> 26625 T23 C26 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26626 T23 C26 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_11,generation=37}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_12,generation=38}
   [junit4]   2> 26627 T23 C26 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 38
   [junit4]   2> 26628 T23 C26 oass.SolrIndexSearcher.<init> Opening Searcher@214f214f main
   [junit4]   2> 26629 T23 C26 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26629 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@214f214f main{StandardDirectoryReader(segments_12:136:nrt)}
   [junit4]   2> 26631 T23 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 6
   [junit4]   2> 26640 T23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26644 T23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_12,generation=38}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_13,generation=39}
   [junit4]   2> 26645 T23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 39
   [junit4]   2> 26647 T23 oass.SolrIndexSearcher.<init> Opening Searcher@2dbc2dbc main
   [junit4]   2> 26648 T23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26648 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dbc2dbc main{StandardDirectoryReader(segments_13:138:nrt _10(4.7):C3)}
   [junit4]   2> 26649 T23 oas.SolrTestCaseJ4.tearDown ###Ending testDupeAllFieldsDetection
   [junit4]   2> 26660 T23 oas.SolrTestCaseJ4.setUp ###Starting testNonStringFieldsValues
   [junit4]   2> 26669 T23 C26 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 26670 T23 C26 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 1
   [junit4]   2> ASYNC  NEW_CORE C27 name=collection1 org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 26684 T23 C27 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26686 T23 C27 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_13,generation=39}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_14,generation=40}
   [junit4]   2> 26686 T23 C27 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 40
   [junit4]   2> 26687 T23 C27 oass.SolrIndexSearcher.<init> Opening Searcher@53b753b7 main
   [junit4]   2> 26688 T23 C27 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26688 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53b753b7 main{StandardDirectoryReader(segments_14:139:nrt)}
   [junit4]   2> 26690 T23 C27 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 6
   [junit4]   2> 26698 T23 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 26705 T23 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 26719 T23 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_14,generation=40}
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@524f524f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5e815e81)),segFN=segments_15,generation=41}
   [junit4]   2> 26720 T23 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 41
   [junit4]   2> 26729 T23 oass.SolrIndexSearcher.<init> Opening Searcher@3db83db8 main
   [junit4]   2> 26730 T23 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 26730 T25 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3db83db8 main{StandardDirectoryReader(segments_15:141:nrt _11(4.7):C5/1:delGen=1)}
   [junit4]   2> 26731 T23 oas.SolrTestCaseJ4.tearDown ###Ending testNonStringFieldsValues
   [junit4]   2> 26738 T23 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 26739 T23 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1998223130
   [junit4]   2> 26739 T23 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7ec67ec6
   [junit4]   2> 26740 T23 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=42,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=253,cumulative_deletesById=0,cumulative_deletesByQuery=6,cumulative_errors=0}
   [junit4]   2> 26741 T23 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 26741 T23 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 26742 T23 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 26743 T23 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 26743 T23 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 26744 T23 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index;done=false>>]
   [junit4]   2> 26744 T23 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658/index
   [junit4]   2> 26745 T23 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658;done=false>>]
   [junit4]   2> 26746 T23 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-SignatureUpdateProcessorFactoryTest-1385315881658
   [junit4]   2> NOTE: test params are: codec=Lucene46: {name=Pulsing41(freqCutoff=10 minBlockSize=76 maxBlockSize=155), ints_is=PostingsFormat(name=MockRandom), weight=PostingsFormat(name=Asserting), id=Pulsing41(freqCutoff=10 minBlockSize=76 maxBlockSize=155), v_t=Pulsing41(freqCutoff=10 minBlockSize=76 maxBlockSize=155), signatureField=PostingsFormat(name=SimpleText)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=pt, timezone=Etc/GMT+8
   [junit4]   2> NOTE: Linux 3.8.0-33-generic amd64/IBM Corporation 1.6.0 (64-bit)/cpus=8,threads=1,free=13115768,total=32936960
   [junit4]   2> NOTE: All tests run in this JVM: [TestRangeQuery, SignatureUpdateProcessorFactoryTest]
   [junit4] Completed on J1 in 4.57s, 6 tests, 1 failure <<< FAILURES!

[...truncated 1042 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:419: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:492: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: There were test failures: 345 suites, 1505 tests, 1 failure, 67 ignored (7 assumptions)

Total time: 38 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/ibm-j9-jdk6 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.7.0_45) - Build # 8351 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8351/
Java: 64bit/jdk1.7.0_45 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Can not find: /onenodecollectioncore/update</title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /onenodecollectioncore/update. Reason: <pre>    Can not find: /onenodecollectioncore/update</pre></p> <hr /><i><small>Powered by Jetty://</small></i>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Can not find: /onenodecollectioncore/update</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /onenodecollectioncore/update. Reason:
<pre>    Can not find: /onenodecollectioncore/update</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([2B68A118C822DA20:AA8E2F00BF7DBA1C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:455)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10077 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
   [junit4]   2> 90174 T255 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 90180 T255 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1385309935009
   [junit4]   2> 90181 T255 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 90181 T256 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 90281 T255 oasc.ZkTestServer.run start zk server on port:57785
   [junit4]   2> 90282 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90374 T262 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c2cb5dc name:ZooKeeperConnection Watcher:127.0.0.1:57785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90375 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90376 T255 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 90385 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90389 T264 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9f052ff name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90389 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90390 T255 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 90395 T255 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 90399 T255 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 90402 T255 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 90406 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 90407 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 90413 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 90414 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 90418 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 90422 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 90428 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 90428 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 90431 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 90432 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 90435 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 90436 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 90439 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 90439 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 90443 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 90443 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 90447 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 90447 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 90451 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 90451 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 90455 T255 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 90455 T255 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 90543 T255 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 90546 T255 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33531
   [junit4]   2> 90547 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 90547 T255 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 90548 T255 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288
   [junit4]   2> 90548 T255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/'
   [junit4]   2> 90558 T255 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/solr.xml
   [junit4]   2> 90594 T255 oasc.CoreContainer.<init> New CoreContainer 773416844
   [junit4]   2> 90594 T255 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/]
   [junit4]   2> 90595 T255 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 90596 T255 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 90596 T255 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 90596 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 90596 T255 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 90597 T255 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 90597 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 90597 T255 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 90597 T255 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 90601 T255 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 90601 T255 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 90602 T255 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 90602 T255 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57785/solr
   [junit4]   2> 90602 T255 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 90603 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90608 T275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c470feb name:ZooKeeperConnection Watcher:127.0.0.1:57785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90609 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90612 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 90614 T277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e26faf2 name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 90614 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 90617 T255 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 90623 T255 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 90629 T255 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 90632 T255 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33531_
   [junit4]   2> 90634 T255 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33531_
   [junit4]   2> 90641 T255 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 90653 T255 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 90658 T255 oasc.Overseer.start Overseer (id=90787671900946435-127.0.0.1:33531_-n_0000000000) starting
   [junit4]   2> 90665 T255 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 90672 T279 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 90673 T255 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 90680 T255 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 90683 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 90690 T278 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 90696 T280 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 90696 T280 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 90698 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 90699 T280 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 90702 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 90704 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33531",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33531_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 90704 T278 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 90708 T278 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 90713 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 90715 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 91699 T280 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 91700 T280 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/collection1
   [junit4]   2> 91700 T280 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 91702 T280 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 91702 T280 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 91705 T280 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/collection1/'
   [junit4]   2> 91706 T280 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/collection1/lib/classes/' to classloader
   [junit4]   2> 91707 T280 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/collection1/lib/README' to classloader
   [junit4]   2> 91761 T280 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 91794 T280 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 91797 T280 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 91803 T280 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 92134 T280 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 92134 T280 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 92135 T280 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 92141 T280 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 92143 T280 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 92170 T280 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 92174 T280 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 92179 T280 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 92182 T280 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 92182 T280 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 92183 T280 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 92185 T280 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 92186 T280 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 92186 T280 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 92186 T280 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1385309935288/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/control/data/
   [junit4]   2> 92186 T280 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6932cb
   [junit4]   2> 92188 T280 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/control/data
   [junit4]   2> 92188 T280 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/control/data/index/
   [junit4]   2> 92188 T280 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 92188 T280 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/control/data/index
   [junit4]   2> 92189 T280 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7331454627779136]
   [junit4]   2> 92190 T280 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5dc98c7c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1dbd2346),segFN=segments_1,generation=1}
   [junit4]   2> 92190 T280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 92194 T280 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 92195 T280 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 92195 T280 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 92195 T280 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 92195 T280 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 92196 T280 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 92196 T280 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 92196 T280 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 92196 T280 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 92197 T280 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 92197 T280 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 92197 T280 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 92198 T280 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 92198 T280 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 92198 T280 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 92199 T280 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 92201 T280 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 92203 T280 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 92203 T280 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 92204 T280 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 92205 T280 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@5dc98c7c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1dbd2346),segFN=segments_1,generation=1}
   [junit4]   2> 92205 T280 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 92205 T280 oass.SolrIndexSearcher.<init> Opening Searcher@73ef6a08 main
   [junit4]   2> 92208 T281 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@73ef6a08 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 92209 T280 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 92209 T280 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33531 collection:control_collection shard:shard1
   [junit4]   2> 92210 T280 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 92223 T280 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 92227 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92229 T280 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 92229 T280 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@136d504b url=http://127.0.0.1:33531/collection1 node=127.0.0.1:33531_ C35_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:33531, core=collection1, node_name=127.0.0.1:33531_}
   [junit4]   2> 92229 T280 C35 P33531 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33531/collection1/
   [junit4]   2> 92230 T280 C35 P33531 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 92230 T280 C35 P33531 oasc.SyncStrategy.syncToMe http://127.0.0.1:33531/collection1/ has no replicas
   [junit4]   2> 92231 T280 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33531/collection1/ shard1
   [junit4]   2> 92231 T280 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 92231 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 92237 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92240 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 92248 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 92252 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92356 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 92396 T280 oasc.ZkController.register We are http://127.0.0.1:33531/collection1/ and leader is http://127.0.0.1:33531/collection1/
   [junit4]   2> 92396 T280 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33531
   [junit4]   2> 92397 T280 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 92397 T280 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 92397 T280 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 92399 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92400 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92400 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92405 T280 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 92407 T255 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 92408 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 92408 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 92410 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33531",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:33531_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 92415 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 92421 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 92424 T284 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@589367ce name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 92425 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 92428 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 92436 T255 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 92513 T255 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 92516 T255 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59429
   [junit4]   2> 92517 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 92517 T255 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 92518 T255 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266
   [junit4]   2> 92519 T255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/'
   [junit4]   2> 92519 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 92519 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 92535 T255 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/solr.xml
   [junit4]   2> 92569 T255 oasc.CoreContainer.<init> New CoreContainer 470593073
   [junit4]   2> 92569 T255 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/]
   [junit4]   2> 92570 T255 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 92570 T255 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 92571 T255 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 92571 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 92571 T255 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 92571 T255 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 92572 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 92572 T255 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 92572 T255 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 92576 T255 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 92577 T255 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 92577 T255 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 92577 T255 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57785/solr
   [junit4]   2> 92577 T255 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 92578 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 92580 T295 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39481d7d name:ZooKeeperConnection Watcher:127.0.0.1:57785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 92581 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 92586 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 92588 T297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d85497a name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 92588 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 92595 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 93602 T255 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59429_
   [junit4]   2> 93603 T255 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59429_
   [junit4]   2> 93607 T277 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 93607 T297 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 93607 T284 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 93614 T298 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 93614 T298 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 93616 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 93616 T298 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 93616 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 93617 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 93619 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 93620 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59429",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:59429_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 93620 T278 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 93621 T278 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 93624 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 93727 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 93727 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 93727 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 94617 T298 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 94617 T298 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/collection1
   [junit4]   2> 94617 T298 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 94618 T298 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 94619 T298 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 94620 T298 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/collection1/'
   [junit4]   2> 94620 T298 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/collection1/lib/classes/' to classloader
   [junit4]   2> 94621 T298 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/collection1/lib/README' to classloader
   [junit4]   2> 94652 T298 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 94683 T298 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 94685 T298 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 94691 T298 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 95010 T298 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 95010 T298 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 95011 T298 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 95017 T298 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 95020 T298 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 95038 T298 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 95042 T298 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 95045 T298 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 95046 T298 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 95047 T298 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 95047 T298 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 95048 T298 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 95048 T298 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 95048 T298 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 95049 T298 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1385309937266/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty1/
   [junit4]   2> 95049 T298 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6932cb
   [junit4]   2> 95049 T298 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty1
   [junit4]   2> 95050 T298 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty1/index/
   [junit4]   2> 95050 T298 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 95050 T298 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty1/index
   [junit4]   2> 95050 T298 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7331454627779136]
   [junit4]   2> 95051 T298 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@70480c1e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@537f9525),segFN=segments_1,generation=1}
   [junit4]   2> 95051 T298 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 95056 T298 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 95056 T298 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 95056 T298 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 95057 T298 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 95057 T298 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 95057 T298 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 95057 T298 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 95058 T298 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 95058 T298 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 95059 T298 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 95059 T298 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 95059 T298 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 95060 T298 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 95060 T298 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 95061 T298 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 95061 T298 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 95063 T298 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 95065 T298 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 95066 T298 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 95066 T298 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 95067 T298 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@70480c1e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@537f9525),segFN=segments_1,generation=1}
   [junit4]   2> 95067 T298 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 95068 T298 oass.SolrIndexSearcher.<init> Opening Searcher@2f7f47d5 main
   [junit4]   2> 95074 T299 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f7f47d5 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 95077 T298 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 95077 T298 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59429 collection:collection1 shard:shard2
   [junit4]   2> 95078 T298 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 95088 T298 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 95090 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95090 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95090 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95091 T298 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 95091 T298 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C36 name=collection1 org.apache.solr.core.SolrCore@7e70c691 url=http://127.0.0.1:59429/collection1 node=127.0.0.1:59429_ C36_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:59429, core=collection1, node_name=127.0.0.1:59429_}
   [junit4]   2> 95092 T298 C36 P59429 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59429/collection1/
   [junit4]   2> 95092 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 95092 T298 C36 P59429 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 95092 T298 C36 P59429 oasc.SyncStrategy.syncToMe http://127.0.0.1:59429/collection1/ has no replicas
   [junit4]   2> 95093 T298 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59429/collection1/ shard2
   [junit4]   2> 95093 T298 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 95095 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95098 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95101 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95204 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95204 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95204 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95249 T298 oasc.ZkController.register We are http://127.0.0.1:59429/collection1/ and leader is http://127.0.0.1:59429/collection1/
   [junit4]   2> 95250 T298 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59429
   [junit4]   2> 95250 T298 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 95250 T298 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 95250 T298 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 95252 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95252 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95252 T298 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 95252 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95254 T255 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 95254 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 95254 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 95255 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:59429",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:59429_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 95258 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 95338 T255 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 95341 T255 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48585
   [junit4]   2> 95342 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 95342 T255 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 95342 T255 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084
   [junit4]   2> 95342 T255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/'
   [junit4]   2> 95352 T255 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/solr.xml
   [junit4]   2> 95360 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95360 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95360 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 95389 T255 oasc.CoreContainer.<init> New CoreContainer 1191534766
   [junit4]   2> 95390 T255 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/]
   [junit4]   2> 95390 T255 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 95391 T255 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 95391 T255 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 95391 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 95392 T255 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 95392 T255 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 95392 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 95392 T255 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 95393 T255 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 95397 T255 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 95397 T255 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 95397 T255 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 95398 T255 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57785/solr
   [junit4]   2> 95398 T255 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 95399 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 95400 T311 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47fbeb99 name:ZooKeeperConnection Watcher:127.0.0.1:57785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 95401 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 95404 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 95405 T313 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@275cd942 name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 95406 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 95409 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 96413 T255 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48585_
   [junit4]   2> 96415 T255 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48585_
   [junit4]   2> 96422 T277 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 96422 T284 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 96422 T313 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 96422 T297 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 96433 T314 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 96433 T314 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 96435 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 96435 T314 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 96436 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 96436 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 96437 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 96438 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48585",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:48585_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 96438 T278 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 96439 T278 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 96442 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 96545 T313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 96545 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 96545 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 96545 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 97436 T314 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 97436 T314 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/collection1
   [junit4]   2> 97436 T314 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 97437 T314 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 97438 T314 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 97439 T314 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/collection1/'
   [junit4]   2> 97440 T314 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/collection1/lib/classes/' to classloader
   [junit4]   2> 97440 T314 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/collection1/lib/README' to classloader
   [junit4]   2> 97475 T314 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 97509 T314 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 97511 T314 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 97518 T314 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 97867 T314 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 97867 T314 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 97868 T314 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 97875 T314 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 97877 T314 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 97894 T314 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 97897 T314 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 97900 T314 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 97901 T314 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 97901 T314 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 97902 T314 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 97903 T314 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 97903 T314 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 97903 T314 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 97903 T314 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1385309940084/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty2/
   [junit4]   2> 97904 T314 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c6932cb
   [junit4]   2> 97905 T314 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty2
   [junit4]   2> 97905 T314 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty2/index/
   [junit4]   2> 97905 T314 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 97905 T314 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1385309935009/jetty2/index
   [junit4]   2> 97906 T314 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7331454627779136]
   [junit4]   2> 97906 T314 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@79459180 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67dc87d9),segFN=segments_1,generation=1}
   [junit4]   2> 97907 T314 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 97913 T314 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 97914 T314 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 97914 T314 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 97914 T314 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 97915 T314 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 97915 T314 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 97915 T314 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 97915 T314 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 97916 T314 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 97916 T314 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 97917 T314 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 97917 T314 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 97917 T314 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 97918 T314 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 97918 T314 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 97919 T314 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 97921 T314 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 97923 T314 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 97923 T314 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 97924 T314 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=41, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 97925 T314 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@79459180 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67dc87d9),segFN=segments_1,generation=1}
   [junit4]   2> 97925 T314 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 97926 T314 oass.SolrIndexSearcher.<init> Opening Searcher@52740d2a main
   [junit4]   2> 97929 T315 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52740d2a main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 97931 T314 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 97931 T314 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48585 collection:collection1 shard:shard1
   [junit4]   2> 97932 T314 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 97939 T314 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 97941 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 97941 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 97941 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 97942 T314 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 97942 T314 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C37 name=collection1 org.apache.solr.core.SolrCore@a72bdef url=http://127.0.0.1:48585/collection1 node=127.0.0.1:48585_ C37_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:48585, core=collection1, node_name=127.0.0.1:48585_}
   [junit4]   2> 97942 T314 C37 P48585 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:48585/collection1/
   [junit4]   2> 97943 T314 C37 P48585 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 97943 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 97943 T314 C37 P48585 oasc.SyncStrategy.syncToMe http://127.0.0.1:48585/collection1/ has no replicas
   [junit4]   2> 97944 T314 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:48585/collection1/ shard1
   [junit4]   2> 97944 T314 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 97948 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 97951 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 97955 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 98058 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98058 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98058 T313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98058 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98102 T314 oasc.ZkController.register We are http://127.0.0.1:48585/collection1/ and leader is http://127.0.0.1:48585/collection1/
   [junit4]   2> 98103 T314 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48585
   [junit4]   2> 98103 T314 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 98103 T314 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 98104 T314 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 98106 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 98106 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 98106 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 98106 T314 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 98108 T255 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 98109 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 98109 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 98110 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48585",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:48585_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 98115 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 98185 T255 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 98190 T255 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48504
   [junit4]   2> 98191 T255 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 98192 T255 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 98193 T255 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939
   [junit4]   2> 98193 T255 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/'
   [junit4]   2> 98213 T255 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/solr.xml
   [junit4]   2> 98218 T313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98218 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98218 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98218 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 98255 T255 oasc.CoreContainer.<init> New CoreContainer 1257296558
   [junit4]   2> 98255 T255 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/]
   [junit4]   2> 98256 T255 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 98256 T255 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 98257 T255 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 98257 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 98257 T255 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 98258 T255 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 98258 T255 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 98258 T255 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 98258 T255 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 98262 T255 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 98263 T255 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 98263 T255 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 98263 T255 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:57785/solr
   [junit4]   2> 98264 T255 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 98265 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98267 T327 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@682069e9 name:ZooKeeperConnection Watcher:127.0.0.1:57785 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98267 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98271 T255 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98274 T329 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d1d6439 name:ZooKeeperConnection Watcher:127.0.0.1:57785/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98274 T255 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98280 T255 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 99284 T255 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48504_
   [junit4]   2> 99286 T255 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48504_
   [junit4]   2> 99289 T313 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 99289 T284 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 99289 T277 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 99289 T329 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 99289 T297 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 99296 T330 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 99297 T330 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 99298 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 99298 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 99298 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 99298 T330 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 99300 T278 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 99301 T278 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:48504",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:48504_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 99301 T278 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 99301 T278 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 99304 T277 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 99407 T277 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 99407 T297 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 99407 T284 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 99407 T313 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 99407 T329 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 100299 T330 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 100299 T330 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/collection1
   [junit4]   2> 100300 T330 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 100301 T330 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 100301 T330 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 100302 T330 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/collection1/'
   [junit4]   2> 100303 T330 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/collection1/lib/classes/' to classloader
   [junit4]   2> 100303 T330 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1385309942939/collection1/lib/README' to classloader
   [junit4]   2> 100346 T330 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 100384 T330 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 100385 T330 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 100392 T330 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 100818 T330 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 100819 T330 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 100819 T330 oass.OpenExchangeRatesOrg

[...truncated too long message...]

": 0.5,
   [junit4]   1>             "MXN": 2.0
   [junit4]   1>           }
   [junit4]   1>         }
   [junit4]   1>         
   [junit4]   1>     /solr/configs/conf1/currency.xml (0)
   [junit4]   1>     DATA: ...supressed...
   [junit4]   1>   /solr/clusterstate.json (0)
   [junit4]   1>   DATA:
   [junit4]   1>       {
   [junit4]   1>         "onenodecollection":{
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:59429",
   [junit4]   1>                   "core":"onenodecollectioncore",
   [junit4]   1>                   "roles":"none",
   [junit4]   1>                   "node_name":"127.0.0.1:59429_",
   [junit4]   1>                   "leader":"true"}}}}},
   [junit4]   1>         "collection1":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:48585",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:48585_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "state":"recovering",
   [junit4]   1>                   "base_url":"http://127.0.0.1:49277",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:49277_"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:59429",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:59429_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:48504",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:48504_"}}}}},
   [junit4]   1>         "control_collection":{
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "shards":{"shard1":{
   [junit4]   1>               "range":"80000000-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{"core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:33531",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:33531_",
   [junit4]   1>                   "leader":"true"}}}}}}
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=2B68A118C822DA20 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ru -Dtests.timezone=America/Nipigon -Dtests.file.encoding=UTF-8
   [junit4] ERROR   24.2s J1 | BasicDistributedZk2Test.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html>
   [junit4]    > <head>
   [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
   [junit4]    > <title>Error 404 Can not find: /onenodecollectioncore/update</title>
   [junit4]    > </head>
   [junit4]    > <body>
   [junit4]    > <h2>HTTP ERROR: 404</h2>
   [junit4]    > <p>Problem accessing /onenodecollectioncore/update. Reason:
   [junit4]    > <pre>    Can not find: /onenodecollectioncore/update</pre></p>
   [junit4]    > <hr /><i><small>Powered by Jetty://</small></i>
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    > </body>
   [junit4]    > </html>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2B68A118C822DA20:AA8E2F00BF7DBA1C]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:455)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
   [junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:198)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 114346 T255 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 24179 T254 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 115413 T346 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=ru, timezone=America/Nipigon
   [junit4]   2> NOTE: Linux 3.8.0-33-generic amd64/Oracle Corporation 1.7.0_45 (64-bit)/cpus=8,threads=1,free=35384856,total=126484480
   [junit4]   2> NOTE: All tests run in this JVM: [HdfsCollectionsAPIDistributedZkTest, DirectSolrConnectionTest, TestArbitraryIndexDir, TestUpdate, TimeZoneUtilsTest, SolrIndexConfigTest, LeaderElectionIntegrationTest, ChangedSchemaMergeTest, TestFastLRUCache, BJQParserTest, TestBinaryResponseWriter, TestSolrCoreProperties, AddBlockUpdateTest, DefaultValueUpdateProcessorTest, TestMultiCoreConfBootstrap, TestDefaultSimilarityFactory, BadIndexSchemaTest, TestCSVLoader, DistributedDebugComponentTest, BasicDistributedZk2Test]
   [junit4] Completed on J1 in 25.46s, 1 test, 1 error <<< FAILURES!

[...truncated 961 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:492: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: There were test failures: 345 suites, 1505 tests, 1 error, 35 ignored (7 assumptions)

Total time: 45 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0_45 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_45) - Build # 8350 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8350/
Java: 32bit/jdk1.7.0_45 -server -XX:+UseParallelGC

3 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch

Error Message:
Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 Can not find: /onenodecollectioncore/update</title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /onenodecollectioncore/update. Reason: <pre>    Can not find: /onenodecollectioncore/update</pre></p> <hr /><i><small>Powered by Jetty://</small></i>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 Can not find: /onenodecollectioncore/update</title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /onenodecollectioncore/update. Reason:
<pre>    Can not find: /onenodecollectioncore/update</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
                                                
</body>
</html>

	at __randomizedtesting.SeedInfo.seed([90A53656172D6D11:1143B84E60720D2D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:455)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch

Error Message:
Still found shard2 in collection test_unload_shard_and_collection

Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
	at __randomizedtesting.SeedInfo.seed([90A53656172D6D11:1143B84E60720D2D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:128)
	at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:78)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)


FAILED:  org.apache.solr.core.TestSolrXmlPersistor.simpleCoreDescriptorIsPersisted

Error Message:
expected:<...r><cores>     <core [instanceDir="instance/dir/" name="testcore]"/> </cores></solr>> but was:<...r><cores>     <core [name="testcore" instanceDir="instance/dir/]"/> </cores></solr>>

Stack Trace:
org.junit.ComparisonFailure: expected:<...r><cores>
    <core [instanceDir="instance/dir/" name="testcore]"/>
</cores></solr>> but was:<...r><cores>
    <core [name="testcore" instanceDir="instance/dir/]"/>
</cores></solr>>
	at __randomizedtesting.SeedInfo.seed([90A53656172D6D11:F0D59624BF9FF983]:0)
	at org.junit.Assert.assertEquals(Assert.java:125)
	at org.junit.Assert.assertEquals(Assert.java:147)
	at org.apache.solr.core.TestSolrXmlPersistor.simpleCoreDescriptorIsPersisted(TestSolrXmlPersistor.java:81)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 10140 lines...]
   [junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest
   [junit4]   2> 181627 T1083 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /hyj/ww
   [junit4]   2> 181635 T1083 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946
   [junit4]   2> 181636 T1083 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 181636 T1084 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 181736 T1083 oasc.ZkTestServer.run start zk server on port:38188
   [junit4]   2> 181737 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 181899 T1090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@aa46ca name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181899 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 181900 T1083 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 181907 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 181911 T1092 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e7acba name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 181911 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 181912 T1083 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 181914 T1083 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 181916 T1083 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 181918 T1083 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 181921 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 181921 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 181931 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 181932 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 181939 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181939 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 181943 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 181943 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 181946 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 181946 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 181948 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 181949 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 181951 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 181952 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 181954 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 181955 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 181957 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181958 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 181961 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 181961 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 181964 T1083 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 181964 T1083 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 181968 T1085 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x1428a8f143c0001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 
   [junit4]   2> 182057 T1083 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 182065 T1083 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51936
   [junit4]   2> 182066 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 182066 T1083 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 182067 T1083 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278
   [junit4]   2> 182067 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278/'
   [junit4]   2> 182080 T1083 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278/solr.xml
   [junit4]   2> 182110 T1083 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 182112 T1083 oasc.CoreContainer.<init> New CoreContainer 5878955
   [junit4]   2> 182112 T1083 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278/]
   [junit4]   2> 182113 T1083 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 182114 T1083 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 182114 T1083 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 182114 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 182114 T1083 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 182115 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 182115 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 182115 T1083 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 182116 T1083 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 182119 T1083 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 182120 T1083 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 182120 T1083 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 182120 T1083 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38188/solr
   [junit4]   2> 182120 T1083 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 182121 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 182123 T1104 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1af8d86 name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 182123 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 182127 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 182127 T1106 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@141a9e name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 182128 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 182129 T1083 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 182141 T1083 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 182144 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 182146 T1083 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51936_hyj%2Fww
   [junit4]   2> 182150 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51936_hyj%2Fww
   [junit4]   2> 182154 T1083 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 182159 T1083 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 182163 T1083 oasc.Overseer.start Overseer (id=90787289625264131-127.0.0.1:51936_hyj%2Fww-n_0000000000) starting
   [junit4]   2> 182166 T1083 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 182171 T1108 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 182172 T1083 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 182174 T1083 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 182176 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 182179 T1107 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 182180 T1083 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278
   [junit4]   2> 182183 T1083 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1385304102278/conf/
   [junit4]   2> 182224 T1083 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 182225 T1083 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 182225 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 182229 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 182230 T1111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16e5880 name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 182230 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 182231 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 182234 T1083 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 182315 T1083 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 182317 T1083 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:42050
   [junit4]   2> 182317 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 182318 T1083 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 182318 T1083 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544
   [junit4]   2> 182318 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/'
   [junit4]   2> 182328 T1083 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/solr.xml
   [junit4]   2> 182363 T1083 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 182363 T1083 oasc.CoreContainer.<init> New CoreContainer 8913029
   [junit4]   2> 182363 T1083 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/]
   [junit4]   2> 182364 T1083 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 182365 T1083 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 182365 T1083 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 182365 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 182365 T1083 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 182366 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 182366 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 182367 T1083 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 182367 T1083 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 182372 T1083 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 182373 T1083 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 182373 T1083 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 182374 T1083 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38188/solr
   [junit4]   2> 182374 T1083 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 182376 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 182377 T1122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f61e0c name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 182377 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 182380 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 182390 T1124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1775952 name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 182390 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 182394 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 183401 T1083 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42050_hyj%2Fww
   [junit4]   2> 183402 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42050_hyj%2Fww
   [junit4]   2> 183405 T1124 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 183406 T1111 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 183406 T1106 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 183410 T1083 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544
   [junit4]   2> 183411 T1083 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/conf/
   [junit4]   2> 183440 T1083 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 183440 T1083 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 183441 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 183528 T1083 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 183530 T1083 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49700
   [junit4]   2> 183530 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 183531 T1083 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 183531 T1083 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752
   [junit4]   2> 183531 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/'
   [junit4]   2> 183541 T1083 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/solr.xml
   [junit4]   2> 183574 T1083 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 183575 T1083 oasc.CoreContainer.<init> New CoreContainer 13533519
   [junit4]   2> 183575 T1083 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/]
   [junit4]   2> 183576 T1083 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 183576 T1083 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 183577 T1083 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 183577 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 183577 T1083 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 183577 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 183578 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 183578 T1083 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 183578 T1083 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 183582 T1083 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 183582 T1083 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 183582 T1083 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 183583 T1083 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38188/solr
   [junit4]   2> 183583 T1083 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 183584 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 183585 T1136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129b15e name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183585 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 183589 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 183592 T1138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a9e51e name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 183592 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 183596 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 184599 T1083 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49700_hyj%2Fww
   [junit4]   2> 184601 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49700_hyj%2Fww
   [junit4]   2> 184604 T1124 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 184604 T1138 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 184604 T1111 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 184604 T1106 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 184609 T1083 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752
   [junit4]   2> 184610 T1083 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/conf/
   [junit4]   2> 184636 T1083 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 184637 T1083 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 184637 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 184716 T1083 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 184719 T1083 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:47034
   [junit4]   2> 184719 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 184719 T1083 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 184720 T1083 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948
   [junit4]   2> 184720 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948/'
   [junit4]   2> 184730 T1083 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948/solr.xml
   [junit4]   2> 184766 T1083 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 184767 T1083 oasc.CoreContainer.<init> New CoreContainer 5188358
   [junit4]   2> 184767 T1083 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948/]
   [junit4]   2> 184769 T1083 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 184769 T1083 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 184769 T1083 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 184770 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 184770 T1083 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 184771 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 184771 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 184772 T1083 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 184772 T1083 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 184777 T1083 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 184778 T1083 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 184778 T1083 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 184778 T1083 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38188/solr
   [junit4]   2> 184779 T1083 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 184782 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 184783 T1150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9f9db name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 184783 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 184785 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 184787 T1152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ae84a4 name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 184787 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 184790 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 185793 T1083 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47034_hyj%2Fww
   [junit4]   2> 185795 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47034_hyj%2Fww
   [junit4]   2> 185798 T1152 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 185798 T1124 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 185798 T1106 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 185799 T1138 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 185799 T1111 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 185806 T1083 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948
   [junit4]   2> 185807 T1083 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1385304104948/conf/
   [junit4]   2> 185838 T1083 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 185839 T1083 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 185839 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 185925 T1083 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 185928 T1083 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36294
   [junit4]   2> 185929 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 185929 T1083 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 185929 T1083 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151
   [junit4]   2> 185930 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151/'
   [junit4]   2> 185941 T1083 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151/solr.xml
   [junit4]   2> 185975 T1083 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 185976 T1083 oasc.CoreContainer.<init> New CoreContainer 32524867
   [junit4]   2> 185976 T1083 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151/]
   [junit4]   2> 185977 T1083 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 185977 T1083 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 185977 T1083 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 185978 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 185978 T1083 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 185978 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 185978 T1083 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 185979 T1083 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 185979 T1083 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 185982 T1083 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 185983 T1083 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 185983 T1083 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 185983 T1083 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:38188/solr
   [junit4]   2> 185984 T1083 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 185984 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 185986 T1164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2aa40 name:ZooKeeperConnection Watcher:127.0.0.1:38188 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185987 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 185991 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 185992 T1166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1078c14 name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185992 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 185996 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 186999 T1083 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36294_hyj%2Fww
   [junit4]   2> 187001 T1083 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36294_hyj%2Fww
   [junit4]   2> 187004 T1106 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187004 T1152 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187004 T1138 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187005 T1111 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187005 T1124 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187005 T1166 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 187010 T1083 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151
   [junit4]   2> 187011 T1083 oasc.CorePropertiesLocator.discoverUnder Found core conf in /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1385304106151/conf/
   [junit4]   2> 187047 T1083 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 187048 T1083 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 187048 T1083 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 187055 T1119 oasc.ZkController.publish publishing core=unloadcollection1 state=down
   [junit4]   2> 187056 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 187056 T1119 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 187058 T1107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 187059 T1107 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42050/hyj/ww",
   [junit4]   2> 	  "core":"unloadcollection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42050_hyj%2Fww",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"unloadcollection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 187059 T1107 oasc.Overseer$ClusterStateUpdater.createCollection Create collection unloadcollection with shards [shard1]
   [junit4]   2> 187059 T1107 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 187064 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 187065 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 187065 T1106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 187065 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 187065 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 187065 T1111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 187069 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188057 T1119 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for unloadcollection1
   [junit4]   2> 188058 T1119 oasc.CoreContainer.create Creating SolrCore 'unloadcollection1' using instanceDir: ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/unloadcollection1
   [junit4]   2> 188059 T1119 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
   [junit4]   2> 188060 T1119 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:unloadcollection
   [junit4]   2> 188060 T1119 oasc.ZkController.getConfName Looking for collection configName
   [junit4]   2> 188061 T1119 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
   [junit4]   2> 188062 T1119 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection
   [junit4]   2> 188064 T1119 oascc.ZkStateReader.readConfigName Load collection config from:/collections/unloadcollection
   [junit4]   2> 188065 T1119 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/unloadcollection1/'
   [junit4]   2> 188116 T1119 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 188157 T1119 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 188158 T1119 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 188166 T1119 oass.IndexSchema.readSchema [unloadcollection1] Schema name=test
   [junit4]   2> 188523 T1119 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 188523 T1119 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 188524 T1119 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 188531 T1119 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 188535 T1119 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 188555 T1119 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 188558 T1119 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 188561 T1119 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 188562 T1119 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 188562 T1119 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 188563 T1119 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 188563 T1119 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 188564 T1119 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 188564 T1119 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 188565 T1119 oasc.SolrCore.<init> [unloadcollection1] Opening new SolrCore at ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1385304102544/unloadcollection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n/
   [junit4]   2> 188565 T1119 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18dea32
   [junit4]   2> 188567 T1119 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n
   [junit4]   2> 188567 T1119 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n/index/
   [junit4]   2> 188567 T1119 oasc.SolrCore.initIndex WARN [unloadcollection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n/index' doesn't exist. Creating new index...
   [junit4]   2> 188568 T1119 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n/index
   [junit4]   2> 188568 T1119 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 188569 T1119 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@92d69a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17cafd1)),segFN=segments_1,generation=1}
   [junit4]   2> 188569 T1119 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 188573 T1119 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 188574 T1119 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 188574 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 188575 T1119 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 188575 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 188576 T1119 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 188576 T1119 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 188577 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 188577 T1119 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 188578 T1119 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 188578 T1119 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 188579 T1119 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 188579 T1119 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 188580 T1119 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 188580 T1119 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 188581 T1119 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 188584 T1119 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 188588 T1119 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 188588 T1119 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 188589 T1119 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18671836303751183]
   [junit4]   2> 188589 T1119 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@92d69a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@17cafd1)),segFN=segments_1,generation=1}
   [junit4]   2> 188589 T1119 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 188590 T1119 oass.SolrIndexSearcher.<init> Opening Searcher@368944 main
   [junit4]   2> 188592 T1168 oasc.SolrCore.registerSearcher [unloadcollection1] Registered new searcher Searcher@368944 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 188594 T1119 oasc.CoreContainer.registerCore registering core: unloadcollection1
   [junit4]   2> 188594 T1119 oasc.ZkController.register Register replica - core:unloadcollection1 address:http://127.0.0.1:42050/hyj/ww collection:unloadcollection shard:shard1
   [junit4]   2> 188595 T1119 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leader_elect/shard1/election
   [junit4]   2> 188602 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 188604 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188604 T1119 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 188604 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C182 name=unloadcollection1 org.apache.solr.core.SolrCore@1ddbf05 url=http://127.0.0.1:42050/hyj/ww/unloadcollection1 node=127.0.0.1:42050_hyj%2Fww C182_STATE=coll:unloadcollection core:unloadcollection1 props:{state=down, base_url=http://127.0.0.1:42050/hyj/ww, core=unloadcollection1, node_name=127.0.0.1:42050_hyj%2Fww}
   [junit4]   2> 188605 T1119 C182 P42050 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42050/hyj/ww/unloadcollection1/
   [junit4]   2> 188605 T1107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 188606 T1119 C182 P42050 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 188606 T1119 C182 P42050 oasc.SyncStrategy.syncToMe http://127.0.0.1:42050/hyj/ww/unloadcollection1/ has no replicas
   [junit4]   2> 188607 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42050/hyj/ww/unloadcollection1/ shard1
   [junit4]   2> 188607 T1119 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leaders/shard1
   [junit4]   2> 188609 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188610 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188610 T1111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188610 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188610 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188610 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188611 T1106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188615 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188616 T1107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 188620 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188723 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188723 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188723 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188723 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188723 T1106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188723 T1111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188766 T1119 oasc.ZkController.register We are http://127.0.0.1:42050/hyj/ww/unloadcollection1/ and leader is http://127.0.0.1:42050/hyj/ww/unloadcollection1/
   [junit4]   2> 188767 T1119 oasc.ZkController.register No LogReplay needed for core=unloadcollection1 baseURL=http://127.0.0.1:42050/hyj/ww
   [junit4]   2> 188767 T1119 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 188767 T1119 oasc.ZkController.publish publishing core=unloadcollection1 state=active
   [junit4]   2> 188769 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188769 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188769 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188769 T1119 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 188770 T1107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 188771 T1107 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:42050/hyj/ww",
   [junit4]   2> 	  "core":"unloadcollection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:42050_hyj%2Fww",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"unloadcollection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 188772 T1119 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304107361unloadcollection1_1n&collection=unloadcollection&version=2&numShards=1&name=unloadcollection1&action=CREATE&wt=javabin} status=0 QTime=1720 
   [junit4]   2> 188774 T1083 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 188774 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188777 T1170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2edfd3 name:ZooKeeperConnection Watcher:127.0.0.1:38188/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 188777 T1083 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 188779 T1083 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 188785 T1083 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 188788 T1133 oasc.ZkController.publish publishing core=unloadcollection2 state=down
   [junit4]   2> 188789 T1133 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 188790 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188790 T1133 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 188792 T1107 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49700/hyj/ww",
   [junit4]   2> 	  "core":"unloadcollection2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49700_hyj%2Fww",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"unloadcollection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 188792 T1107 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
   [junit4]   2> 188792 T1107 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 188795 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 188898 T1138 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1106 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 188898 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 189792 T1133 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for unloadcollection2
   [junit4]   2> 189793 T1133 oasc.CoreContainer.create Creating SolrCore 'unloadcollection2' using instanceDir: ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/unloadcollection2
   [junit4]   2> 189793 T1133 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
   [junit4]   2> 189794 T1133 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 189794 T1133 oascc.ZkStateReader.readConfigName Load collection config from:/collections/unloadcollection
   [junit4]   2> 189795 T1133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/unloadcollection2/'
   [junit4]   2> 189828 T1133 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 189862 T1133 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 189864 T1133 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 189870 T1133 oass.IndexSchema.readSchema [unloadcollection2] Schema name=test
   [junit4]   2> 190174 T1133 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 190175 T1133 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 190175 T1133 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 190181 T1133 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 190184 T1133 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 190209 T1133 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 190214 T1133 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 190219 T1133 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 190221 T1133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 190221 T1133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 190221 T1133 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 190224 T1133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 190224 T1133 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 190224 T1133 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 190225 T1133 oasc.SolrCore.<init> [unloadcollection2] Opening new SolrCore at ./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1385304103752/unloadcollection2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n/
   [junit4]   2> 190225 T1133 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18dea32
   [junit4]   2> 190226 T1133 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n
   [junit4]   2> 190227 T1133 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n/index/
   [junit4]   2> 190227 T1133 oasc.SolrCore.initIndex WARN [unloadcollection2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n/index' doesn't exist. Creating new index...
   [junit4]   2> 190227 T1133 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n/index
   [junit4]   2> 190228 T1133 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 190229 T1133 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@991fc0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@80cf56)),segFN=segments_1,generation=1}
   [junit4]   2> 190229 T1133 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 190233 T1133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 190234 T1133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 190234 T1133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 190235 T1133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 190235 T1133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 190235 T1133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 190235 T1133 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 190236 T1133 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 190236 T1133 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 190236 T1133 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 190237 T1133 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 190237 T1133 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 190237 T1133 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 190238 T1133 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 190238 T1133 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 190239 T1133 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 190240 T1133 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 190242 T1133 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 190242 T1133 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 190243 T1133 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.18671836303751183]
   [junit4]   2> 190243 T1133 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(org.apache.lucene.store.RAMDirectory@991fc0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@80cf56)),segFN=segments_1,generation=1}
   [junit4]   2> 190244 T1133 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 190244 T1133 oass.SolrIndexSearcher.<init> Opening Searcher@1532d72 main
   [junit4]   2> 190246 T1171 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@1532d72 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 190248 T1133 oasc.CoreContainer.registerCore registering core: unloadcollection2
   [junit4]   2> 190248 T1133 oasc.ZkController.register Register replica - core:unloadcollection2 address:http://127.0.0.1:49700/hyj/ww collection:unloadcollection shard:shard1
   [junit4]   2> 190251 T1133 oasc.ZkController.register We are http://127.0.0.1:49700/hyj/ww/unloadcollection2/ and leader is http://127.0.0.1:42050/hyj/ww/unloadcollection1/
   [junit4]   2> 190252 T1133 oasc.ZkController.register No LogReplay needed for core=unloadcollection2 baseURL=http://127.0.0.1:49700/hyj/ww
   [junit4]   2> 190252 T1133 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection2
   [junit4]   2> 190253 T1133 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> ASYNC  NEW_CORE C183 name=unloadcollection2 org.apache.solr.core.SolrCore@142b225 url=http://127.0.0.1:49700/hyj/ww/unloadcollection2 node=127.0.0.1:49700_hyj%2Fww C183_STATE=coll:unloadcollection core:unloadcollection2 props:{state=down, base_url=http://127.0.0.1:49700/hyj/ww, core=unloadcollection2, node_name=127.0.0.1:49700_hyj%2Fww}
   [junit4]   2> 190254 T1172 C183 P49700 oasc.RecoveryStrategy.run Starting recovery process.  core=unloadcollection2 recoveringAfterStartup=true
   [junit4]   2> 190254 T1172 C183 P49700 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 190254 T1172 C183 P49700 oasc.ZkController.publish publishing core=unloadcollection2 state=recovering
   [junit4]   2> 190254 T1172 C183 P49700 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 190259 T1133 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 190259 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 190260 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 190260 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 190261 T1107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 190262 T1133 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1385304101946/1385304109096unloadcollection1_2n&collection=unloadcollection&version=2&name=unloadcollection2&action=CREATE&wt=javabin} status=0 QTime=1474 
   [junit4]   2> 190262 T1120 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
   [junit4]   2> 190263 T1083 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 190263 T1107 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:49700/hyj/ww",
   [junit4]   2> 	  "core":"unloadcollection2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:49700_hyj%2Fww",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "shard_range":null,
   [junit4]   2> 	  "shard_state":"active",
   [junit4]   2> 	  "shard_parent":null,
   [junit4]   2> 	  "collection":"unloadcollection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 190264 T1083 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):330
   [junit4]   2> 190276 T1083 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 190280 T1106 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 190383 T1124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 190383 T1166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 190383 T1152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurr

[...truncated too long message...]

  "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:52096_",
   [junit4]   1>                   "leader":"true"}}}},
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "maxShardsPerNode":"1",
   [junit4]   1>           "autoCreated":"true"},
   [junit4]   1>         "collection1":{
   [junit4]   1>           "shards":{
   [junit4]   1>             "shard1":{
   [junit4]   1>               "range":"80000000-ffffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node2":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:39821",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:39821_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node4":{
   [junit4]   1>                   "state":"recovering",
   [junit4]   1>                   "base_url":"http://127.0.0.1:33842",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:33842_"}}},
   [junit4]   1>             "shard2":{
   [junit4]   1>               "range":"0-7fffffff",
   [junit4]   1>               "state":"active",
   [junit4]   1>               "replicas":{
   [junit4]   1>                 "core_node1":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:47121",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:47121_",
   [junit4]   1>                   "leader":"true"},
   [junit4]   1>                 "core_node3":{
   [junit4]   1>                   "state":"active",
   [junit4]   1>                   "base_url":"http://127.0.0.1:37412",
   [junit4]   1>                   "core":"collection1",
   [junit4]   1>                   "node_name":"127.0.0.1:37412_"}}}},
   [junit4]   1>           "replicationFactor":"1",
   [junit4]   1>           "router":{"name":"compositeId"},
   [junit4]   1>           "autoCreated":"true",
   [junit4]   1>           "maxShardsPerNode":"1"}}
   [junit4]   1>   /solr/live_nodes (5)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:39821_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:52096_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:37412_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:33842_ (0)
   [junit4]   1>    /solr/live_nodes/127.0.0.1:47121_ (0)
   [junit4]   1>  /zookeeper (1)
   [junit4]   1>  DATA:
   [junit4]   1>      
   [junit4]   1> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=90A53656172D6D11 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh_SG -Dtests.timezone=US/Central -Dtests.file.encoding=UTF-8
   [junit4] ERROR   23.4s J1 | BasicDistributedZk2Test.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected content type application/octet-stream but got text/html;charset=ISO-8859-1. <html>
   [junit4]    > <head>
   [junit4]    > <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
   [junit4]    > <title>Error 404 Can not find: /onenodecollectioncore/update</title>
   [junit4]    > </head>
   [junit4]    > <body>
   [junit4]    > <h2>HTTP ERROR: 404</h2>
   [junit4]    > <p>Problem accessing /onenodecollectioncore/update. Reason:
   [junit4]    > <pre>    Can not find: /onenodecollectioncore/update</pre></p>
   [junit4]    > <hr /><i><small>Powered by Jetty://</small></i>
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    >                                                 
   [junit4]    > </body>
   [junit4]    > </html>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([90A53656172D6D11:1143B84E60720D2D]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:455)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:197)
   [junit4]    > 	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:196)
   [junit4]    > 	at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:88)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 498853 T1629 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 23383 T1628 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 499054 T1703 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 500142 T1720 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=zh_SG, timezone=US/Central
   [junit4]   2> NOTE: Linux 3.8.0-33-generic i386/Oracle Corporation 1.7.0_45 (32-bit)/cpus=8,threads=3,free=116661880,total=274464768
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterConfigTest, TestReload, TestZkChroot, AssignTest, RAMDirectoryFactoryTest, CoreContainerCoreInitFailuresTest, TestIBSimilarityFactory, TestAddFieldRealTimeGet, SolrCmdDistributorTest, HighlighterTest, ChaosMonkeyNothingIsSafeTest, TestAnalyzeInfixSuggestions, HardAutoCommitTest, PluginInfoTest, TestOmitPositions, PrimUtilsTest, TestFieldTypeCollectionResource, OpenExchangeRatesOrgProviderTest, PreAnalyzedUpdateProcessorTest, SortByFunctionTest, TestRangeQuery, TestQuerySenderNoQuery, StatsComponentTest, TestLazyCores, AddBlockUpdateTest, TestStressRecovery, TestStressReorder, TestDistributedGrouping, InfoHandlerTest, TestCollapseQParserPlugin, BasicZkTest, TestSchemaNameResource, TestQuerySenderListener, PathHierarchyTokenizerFactoryTest, TestSolrJ, BJQParserTest, SuggesterTSTTest, BadIndexSchemaTest, TestHashPartitioner, TestDistributedSearch, TestUtils, ChaosMonkeySafeLeaderTest, CurrencyFieldOpenExchangeTest, SolrRequestParserTest, DistributedDebugComponentTest, SuggesterWFSTTest, TestSchemaVersionResource, TestFuzzyAnalyzedSuggestions, SpellCheckCollatorTest, ConvertedLegacyTest, DirectSolrSpellCheckerTest, TestSchemaSimilarityResource, DebugComponentTest, PingRequestHandlerTest, TestStressUserVersions, ChangedSchemaMergeTest, TestFoldingMultitermQuery, TestFiltering, RequiredFieldsTest, SpellingQueryConverterTest, RequestHandlersTest, ZkNodePropsTest, DocumentAnalysisRequestHandlerTest, TestStressLucene, TestSolrXmlPersistor, CoreAdminCreateDiscoverTest, TestCoreDiscovery, TestRandomMergePolicy, SolrTestCaseJ4Test, TestXIncludeConfig, NotRequiredUniqueKeyTest, BasicDistributedZk2Test]
   [junit4] Completed on J1 in 24.91s, 1 test, 1 error <<< FAILURES!

[...truncated 587 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:492: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: There were test failures: 345 suites, 1505 tests, 1 error, 2 failures, 34 ignored (6 assumptions)

Total time: 45 minutes 35 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_45 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Seán Coffey <se...@oracle.com>.
Bug report is now public : https://bugs.openjdk.java.net/browse/JDK-8029166

regards,
Sean.

On 26/11/2013 11:46, Rory O'Donnell Oracle, Dublin Ireland wrote:
> Hi Uwe,
>
> Thanks for that.
>
> By the way, b117 is now available at: https://jdk8.java.net/download.html
>
> Rgds,Rory
>
>
> On 26/11/2013 11:42, Uwe Schindler wrote:
>> Hi Rory,
>>
>> I filed the bug report: Bug ID: 9008475 - SIGSEGV while garbage 
>> collection in Apache Lucene with ConcMarkSweepGC on JDK 8
>>
>> Thanks,
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
>>> Sent: Monday, November 25, 2013 3:00 PM
>>> To: Uwe Schindler
>>> Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; COFFEY;
>>> balchandra.vaidya@oracle.com; TOPIC,DALIBOR
>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux 
>>> (32bit/jdk1.8.0-ea-b116) - Build
>>> # 8349 - Failure!
>>>
>>> Hi Uwe,
>>>
>>> Can you please let me know the incident number when you report the 
>>> bug ?
>>>
>>> Rgds,Rory
>>> On 25/11/2013 12:38, Uwe Schindler wrote:
>>>> Hi,
>>>>
>>>> In addition, this is a GC bug, so it happens in the GC threads - which
>>> explains why it does not easy reproduce:
>>>> http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/luc
>>>> ene/build/core/test/J0/hs_err_pid11654.log
>>>>
>>>> This one happens in ConcMarkSweepGC!
>>>>
>>>> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>>>>
>>>> Current thread (0xf6a14000):  GCTaskThread [stack:
>>>> 0xf447f000,0xf4500000] [id=11664]
>>>>
>>>> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
>>>> si_addr=0x00000004
>>>>
>>>> Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k
>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
>>>> C=native code) V  [libjvm.so+0x4475f1]
>>>> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
>>>> ParScanWithoutBarrierClosure*)+0x61
>>>> V  [libjvm.so+0x47326f] ParScanThreadState::trim_queues(int)+0xdf
>>>> V  [libjvm.so+0x473762] ParEvacuateFollowersClosure::do_void()+0x22
>>>> V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V
>>>> [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V [libjvm.so+0x58b2c8]
>>>> GangWorker::run()+0x18 V  [libjvm.so+0x461689]
>>>> java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]
>>>> start_thread+0xcc
>>>>
>>>> I think the only thing we can do is to open a bug report!
>>>>
>>>> -----
>>>> Uwe Schindler
>>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>> http://www.thetaphi.de
>>>> eMail: uwe@thetaphi.de
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>>>>> Sent: Monday, November 25, 2013 12:18 PM
>>>>> To: 'dev@lucene.apache.org'
>>>>> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
>>>>> - Build # 8349 - Failure!
>>>>>
>>>>> Same here. I also ran the whole test suite, does not reproduce!
>>>>>
>>>>> -----
>>>>> Uwe Schindler
>>>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>>> http://www.thetaphi.de
>>>>> eMail: uwe@thetaphi.de
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On
>>> Behalf
>>>>>> Of Dawid Weiss
>>>>>> Sent: Monday, November 25, 2013 11:04 AM
>>>>>> To: dev@lucene.apache.org
>>>>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux
>>>>>> (32bit/jdk1.8.0-ea-b116)
>>>>>> - Build # 8349 - Failure!
>>>>>>
>>>>>> I can't reproduce this. I've ran the same (?) test args 20 times, 
>>>>>> no luck.
>>>>>>
>>>>>> Dawid
>>>>>>
>>>>>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
>>>>> wrote:
>>>>>>> Not yet tried!
>>>>>>> I am not at home at the moment.
>>>>>>>
>>>>>>> Uwe
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>>>>>>>>>    this JVM didn't last long ;)
>>>>>>>> If only we got a penny for every JVM crash... :)
>>>>>>>>
>>>>>>>> Uwe -- success at reproducing?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> ________________________________
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>>
>>>>>>> -- 
>>>>>>> Uwe Schindler
>>>>>>> H.-H.-Meier-Allee 63, 28213 Bremen
>>>>>>> http://www.thetaphi.de
>>>>>> --------------------------------------------------------------------
>>>>>> - To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>> -- 
>>> Rgds,Rory O'Donnell
>>> Quality Engineering Manager
>>> Oracle EMEA , Dublin, Ireland
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by "Rory O'Donnell Oracle, Dublin Ireland" <ro...@oracle.com>.
On 26/11/2013 13:04, Uwe Schindler wrote:
> Hi,
>
>> By the way, b117 is now available at: https://jdk8.java.net/download.html
> Thanks, will upgrade in a moment. This still does not allow us to run the whole build, because the javadocs bug with java.text subclasses is not yet fixed (https://bugs.openjdk.java.net/browse/JDK-8027977; I hope it is in the next one). But we can try the fix of Vladimir for the Hotspot bug with Vector CPU Extensions.
Yes ,I hope https://bugs.openjdk.java.net/browse/JDK-8027977 will be in 
b118.

Rgds,Rory
>
> Uwe
>
>> On 26/11/2013 11:42, Uwe Schindler wrote:
>>> Hi Rory,
>>>
>>> I filed the bug report: Bug ID: 9008475 - SIGSEGV while garbage
>>> collection in Apache Lucene with ConcMarkSweepGC on JDK 8
>>>
>>> Thanks,
>>> Uwe
>>>
>>> -----
>>> Uwe Schindler
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>> http://www.thetaphi.de
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>>> -----Original Message-----
>>>> From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
>>>> Sent: Monday, November 25, 2013 3:00 PM
>>>> To: Uwe Schindler
>>>> Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; COFFEY;
>>>> balchandra.vaidya@oracle.com; TOPIC,DALIBOR
>>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
>>>> - Build # 8349 - Failure!
>>>>
>>>> Hi Uwe,
>>>>
>>>> Can you please let me know the incident number when you report the
>> bug ?
>>>> Rgds,Rory
>>>> On 25/11/2013 12:38, Uwe Schindler wrote:
>>>>> Hi,
>>>>>
>>>>> In addition, this is a GC bug, so it happens in the GC threads -
>>>>> which
>>>> explains why it does not easy reproduce:
>>>>> http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/l
>>>>> uc ene/build/core/test/J0/hs_err_pid11654.log
>>>>>
>>>>> This one happens in ConcMarkSweepGC!
>>>>>
>>>>> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>>>>>
>>>>> Current thread (0xf6a14000):  GCTaskThread [stack:
>>>>> 0xf447f000,0xf4500000] [id=11664]
>>>>>
>>>>> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
>>>>> si_addr=0x00000004
>>>>>
>>>>> Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k
>>>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
>>>>> C=native code) V  [libjvm.so+0x4475f1]
>>>>> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
>>>>> ParScanWithoutBarrierClosure*)+0x61
>>>>> V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
>>>>> V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
>>>>> V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V
>>>>> [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V
>>>>> [libjvm.so+0x58b2c8]
>>>>> GangWorker::run()+0x18 V  [libjvm.so+0x461689]
>>>>> java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]
>>>>> start_thread+0xcc
>>>>>
>>>>> I think the only thing we can do is to open a bug report!
>>>>>
>>>>> -----
>>>>> Uwe Schindler
>>>>> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
>>>>> eMail: uwe@thetaphi.de
>>>>>
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>>>>>> Sent: Monday, November 25, 2013 12:18 PM
>>>>>> To: 'dev@lucene.apache.org'
>>>>>> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
>>>>>> - Build # 8349 - Failure!
>>>>>>
>>>>>> Same here. I also ran the whole test suite, does not reproduce!
>>>>>>
>>>>>> -----
>>>>>> Uwe Schindler
>>>>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>>>> http://www.thetaphi.de
>>>>>> eMail: uwe@thetaphi.de
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On
>>>> Behalf
>>>>>>> Of Dawid Weiss
>>>>>>> Sent: Monday, November 25, 2013 11:04 AM
>>>>>>> To: dev@lucene.apache.org
>>>>>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux
>>>>>>> (32bit/jdk1.8.0-ea-b116)
>>>>>>> - Build # 8349 - Failure!
>>>>>>>
>>>>>>> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
>>>>>>>
>>>>>>> Dawid
>>>>>>>
>>>>>>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
>>>>>> wrote:
>>>>>>>> Not yet tried!
>>>>>>>> I am not at home at the moment.
>>>>>>>>
>>>>>>>> Uwe
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>>>>>>>>>>     this JVM didn't last long ;)
>>>>>>>>> If only we got a penny for every JVM crash... :)
>>>>>>>>>
>>>>>>>>> Uwe -- success at reproducing?
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Uwe Schindler
>>>>>>>> H.-H.-Meier-Allee 63, 28213 Bremen
>>>>>>>> http://www.thetaphi.de
>>>>>>> --------------------------------------------------------------------
>>>>>>> - To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>> --
>>>> Rgds,Rory O'Donnell
>>>> Quality Engineering Manager
>>>> Oracle EMEA , Dublin, Ireland
>> --
>> Rgds,Rory O'Donnell
>> Quality Engineering Manager
>> Oracle EMEA , Dublin, Ireland
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

> By the way, b117 is now available at: https://jdk8.java.net/download.html

Thanks, will upgrade in a moment. This still does not allow us to run the whole build, because the javadocs bug with java.text subclasses is not yet fixed (https://bugs.openjdk.java.net/browse/JDK-8027977; I hope it is in the next one). But we can try the fix of Vladimir for the Hotspot bug with Vector CPU Extensions.

Uwe 

> On 26/11/2013 11:42, Uwe Schindler wrote:
> > Hi Rory,
> >
> > I filed the bug report: Bug ID: 9008475 - SIGSEGV while garbage
> > collection in Apache Lucene with ConcMarkSweepGC on JDK 8
> >
> > Thanks,
> > Uwe
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
> >> Sent: Monday, November 25, 2013 3:00 PM
> >> To: Uwe Schindler
> >> Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; COFFEY;
> >> balchandra.vaidya@oracle.com; TOPIC,DALIBOR
> >> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
> >> - Build # 8349 - Failure!
> >>
> >> Hi Uwe,
> >>
> >> Can you please let me know the incident number when you report the
> bug ?
> >>
> >> Rgds,Rory
> >> On 25/11/2013 12:38, Uwe Schindler wrote:
> >>> Hi,
> >>>
> >>> In addition, this is a GC bug, so it happens in the GC threads -
> >>> which
> >> explains why it does not easy reproduce:
> >>> http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/l
> >>> uc ene/build/core/test/J0/hs_err_pid11654.log
> >>>
> >>> This one happens in ConcMarkSweepGC!
> >>>
> >>> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
> >>>
> >>> Current thread (0xf6a14000):  GCTaskThread [stack:
> >>> 0xf447f000,0xf4500000] [id=11664]
> >>>
> >>> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> >>> si_addr=0x00000004
> >>>
> >>> Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k
> >>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> >>> C=native code) V  [libjvm.so+0x4475f1]
> >>> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
> >>> ParScanWithoutBarrierClosure*)+0x61
> >>> V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
> >>> V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
> >>> V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V
> >>> [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V
> >>> [libjvm.so+0x58b2c8]
> >>> GangWorker::run()+0x18 V  [libjvm.so+0x461689]
> >>> java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]
> >>> start_thread+0xcc
> >>>
> >>> I think the only thing we can do is to open a bug report!
> >>>
> >>> -----
> >>> Uwe Schindler
> >>> H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de
> >>> eMail: uwe@thetaphi.de
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> >>>> Sent: Monday, November 25, 2013 12:18 PM
> >>>> To: 'dev@lucene.apache.org'
> >>>> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
> >>>> - Build # 8349 - Failure!
> >>>>
> >>>> Same here. I also ran the whole test suite, does not reproduce!
> >>>>
> >>>> -----
> >>>> Uwe Schindler
> >>>> H.-H.-Meier-Allee 63, D-28213 Bremen
> >>>> http://www.thetaphi.de
> >>>> eMail: uwe@thetaphi.de
> >>>>
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On
> >> Behalf
> >>>>> Of Dawid Weiss
> >>>>> Sent: Monday, November 25, 2013 11:04 AM
> >>>>> To: dev@lucene.apache.org
> >>>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux
> >>>>> (32bit/jdk1.8.0-ea-b116)
> >>>>> - Build # 8349 - Failure!
> >>>>>
> >>>>> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
> >>>>>
> >>>>> Dawid
> >>>>>
> >>>>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
> >>>> wrote:
> >>>>>> Not yet tried!
> >>>>>> I am not at home at the moment.
> >>>>>>
> >>>>>> Uwe
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
> >>>>>>>>    this JVM didn't last long ;)
> >>>>>>> If only we got a penny for every JVM crash... :)
> >>>>>>>
> >>>>>>> Uwe -- success at reproducing?
> >>>>>>>
> >>>>>>> D.
> >>>>>>>
> >>>>>>> ________________________________
> >>>>>>>
> >>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>>>
> >>>>>> --
> >>>>>> Uwe Schindler
> >>>>>> H.-H.-Meier-Allee 63, 28213 Bremen
> >>>>>> http://www.thetaphi.de
> >>>>> --------------------------------------------------------------------
> >>>>> - To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >>>>> additional commands, e-mail: dev-help@lucene.apache.org
> >> --
> >> Rgds,Rory O'Donnell
> >> Quality Engineering Manager
> >> Oracle EMEA , Dublin, Ireland
> 
> --
> Rgds,Rory O'Donnell
> Quality Engineering Manager
> Oracle EMEA , Dublin, Ireland
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by "Rory O'Donnell Oracle, Dublin Ireland" <ro...@oracle.com>.
Hi Uwe,

Thanks for that.

By the way, b117 is now available at: https://jdk8.java.net/download.html

Rgds,Rory


On 26/11/2013 11:42, Uwe Schindler wrote:
> Hi Rory,
>
> I filed the bug report: Bug ID: 9008475 - SIGSEGV while garbage collection in Apache Lucene with ConcMarkSweepGC on JDK 8
>
> Thanks,
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
>> Sent: Monday, November 25, 2013 3:00 PM
>> To: Uwe Schindler
>> Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; COFFEY;
>> balchandra.vaidya@oracle.com; TOPIC,DALIBOR
>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
>> # 8349 - Failure!
>>
>> Hi Uwe,
>>
>> Can you please let me know the incident number when you report the bug ?
>>
>> Rgds,Rory
>> On 25/11/2013 12:38, Uwe Schindler wrote:
>>> Hi,
>>>
>>> In addition, this is a GC bug, so it happens in the GC threads - which
>> explains why it does not easy reproduce:
>>> http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/luc
>>> ene/build/core/test/J0/hs_err_pid11654.log
>>>
>>> This one happens in ConcMarkSweepGC!
>>>
>>> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>>>
>>> Current thread (0xf6a14000):  GCTaskThread [stack:
>>> 0xf447f000,0xf4500000] [id=11664]
>>>
>>> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
>>> si_addr=0x00000004
>>>
>>> Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k
>>> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
>>> C=native code) V  [libjvm.so+0x4475f1]
>>> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
>>> ParScanWithoutBarrierClosure*)+0x61
>>> V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
>>> V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
>>> V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V
>>> [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V  [libjvm.so+0x58b2c8]
>>> GangWorker::run()+0x18 V  [libjvm.so+0x461689]
>>> java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]
>>> start_thread+0xcc
>>>
>>> I think the only thing we can do is to open a bug report!
>>>
>>> -----
>>> Uwe Schindler
>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>> http://www.thetaphi.de
>>> eMail: uwe@thetaphi.de
>>>
>>>
>>>> -----Original Message-----
>>>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>>>> Sent: Monday, November 25, 2013 12:18 PM
>>>> To: 'dev@lucene.apache.org'
>>>> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
>>>> - Build # 8349 - Failure!
>>>>
>>>> Same here. I also ran the whole test suite, does not reproduce!
>>>>
>>>> -----
>>>> Uwe Schindler
>>>> H.-H.-Meier-Allee 63, D-28213 Bremen
>>>> http://www.thetaphi.de
>>>> eMail: uwe@thetaphi.de
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On
>> Behalf
>>>>> Of Dawid Weiss
>>>>> Sent: Monday, November 25, 2013 11:04 AM
>>>>> To: dev@lucene.apache.org
>>>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux
>>>>> (32bit/jdk1.8.0-ea-b116)
>>>>> - Build # 8349 - Failure!
>>>>>
>>>>> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
>>>>>
>>>>> Dawid
>>>>>
>>>>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
>>>> wrote:
>>>>>> Not yet tried!
>>>>>> I am not at home at the moment.
>>>>>>
>>>>>> Uwe
>>>>>>
>>>>>>
>>>>>>
>>>>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>>>>>>>>    this JVM didn't last long ;)
>>>>>>> If only we got a penny for every JVM crash... :)
>>>>>>>
>>>>>>> Uwe -- success at reproducing?
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>> ________________________________
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>>>>>
>>>>>> --
>>>>>> Uwe Schindler
>>>>>> H.-H.-Meier-Allee 63, 28213 Bremen
>>>>>> http://www.thetaphi.de
>>>>> --------------------------------------------------------------------
>>>>> - To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>> --
>> Rgds,Rory O'Donnell
>> Quality Engineering Manager
>> Oracle EMEA , Dublin, Ireland

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi Rory,

I filed the bug report: Bug ID: 9008475 - SIGSEGV while garbage collection in Apache Lucene with ConcMarkSweepGC on JDK 8

Thanks,
Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Rory O'Donnell [mailto:rory.odonnell@oracle.com]
> Sent: Monday, November 25, 2013 3:00 PM
> To: Uwe Schindler
> Cc: dev@lucene.apache.org; rory.odonnell@oracle.com; COFFEY;
> balchandra.vaidya@oracle.com; TOPIC,DALIBOR
> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
> # 8349 - Failure!
> 
> Hi Uwe,
> 
> Can you please let me know the incident number when you report the bug ?
> 
> Rgds,Rory
> On 25/11/2013 12:38, Uwe Schindler wrote:
> > Hi,
> >
> > In addition, this is a GC bug, so it happens in the GC threads - which
> explains why it does not easy reproduce:
> > http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/luc
> > ene/build/core/test/J0/hs_err_pid11654.log
> >
> > This one happens in ConcMarkSweepGC!
> >
> > Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
> >
> > Current thread (0xf6a14000):  GCTaskThread [stack:
> > 0xf447f000,0xf4500000] [id=11664]
> >
> > siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
> > si_addr=0x00000004
> >
> > Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k
> > Native frames: (J=compiled Java code, j=interpreted, Vv=VM code,
> > C=native code) V  [libjvm.so+0x4475f1]
> > ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
> > ParScanWithoutBarrierClosure*)+0x61
> > V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
> > V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
> > V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V
> > [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V  [libjvm.so+0x58b2c8]
> > GangWorker::run()+0x18 V  [libjvm.so+0x461689]
> > java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]
> > start_thread+0xcc
> >
> > I think the only thing we can do is to open a bug report!
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> >> Sent: Monday, November 25, 2013 12:18 PM
> >> To: 'dev@lucene.apache.org'
> >> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
> >> - Build # 8349 - Failure!
> >>
> >> Same here. I also ran the whole test suite, does not reproduce!
> >>
> >> -----
> >> Uwe Schindler
> >> H.-H.-Meier-Allee 63, D-28213 Bremen
> >> http://www.thetaphi.de
> >> eMail: uwe@thetaphi.de
> >>
> >>
> >>> -----Original Message-----
> >>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On
> Behalf
> >>> Of Dawid Weiss
> >>> Sent: Monday, November 25, 2013 11:04 AM
> >>> To: dev@lucene.apache.org
> >>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux
> >>> (32bit/jdk1.8.0-ea-b116)
> >>> - Build # 8349 - Failure!
> >>>
> >>> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
> >>>
> >>> Dawid
> >>>
> >>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
> >> wrote:
> >>>> Not yet tried!
> >>>> I am not at home at the moment.
> >>>>
> >>>> Uwe
> >>>>
> >>>>
> >>>>
> >>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
> >>>>>>   this JVM didn't last long ;)
> >>>>>
> >>>>> If only we got a penny for every JVM crash... :)
> >>>>>
> >>>>> Uwe -- success at reproducing?
> >>>>>
> >>>>> D.
> >>>>>
> >>>>> ________________________________
> >>>>>
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >>>>> additional commands, e-mail: dev-help@lucene.apache.org
> >>>>>
> >>>> --
> >>>> Uwe Schindler
> >>>> H.-H.-Meier-Allee 63, 28213 Bremen
> >>>> http://www.thetaphi.de
> >>> --------------------------------------------------------------------
> >>> - To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >>> additional commands, e-mail: dev-help@lucene.apache.org
> 
> --
> Rgds,Rory O'Donnell
> Quality Engineering Manager
> Oracle EMEA , Dublin, Ireland


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Rory O'Donnell <ro...@oracle.com>.
Hi Uwe,

Can you please let me know the incident number when you report the bug ?

Rgds,Rory
On 25/11/2013 12:38, Uwe Schindler wrote:
> Hi,
>
> In addition, this is a GC bug, so it happens in the GC threads - which explains why it does not easy reproduce:
> http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/lucene/build/core/test/J0/hs_err_pid11654.log
>
> This one happens in ConcMarkSweepGC!
>
> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>
> Current thread (0xf6a14000):  GCTaskThread [stack: 0xf447f000,0xf4500000] [id=11664]
>
> siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000004
>
> Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V  [libjvm.so+0x4475f1]  ObjArrayKlass::oop_oop_iterate_nv(oopDesc*, ParScanWithoutBarrierClosure*)+0x61
> V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
> V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
> V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V  [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V  [libjvm.so+0x58b2c8]  GangWorker::run()+0x18 V  [libjvm.so+0x461689]  java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]  start_thread+0xcc
>
> I think the only thing we can do is to open a bug report!
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> Sent: Monday, November 25, 2013 12:18 PM
>> To: 'dev@lucene.apache.org'
>> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
>> # 8349 - Failure!
>>
>> Same here. I also ran the whole test suite, does not reproduce!
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>>> -----Original Message-----
>>> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf
>>> Of Dawid Weiss
>>> Sent: Monday, November 25, 2013 11:04 AM
>>> To: dev@lucene.apache.org
>>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
>>> - Build # 8349 - Failure!
>>>
>>> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
>>>
>>> Dawid
>>>
>>> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
>> wrote:
>>>> Not yet tried!
>>>> I am not at home at the moment.
>>>>
>>>> Uwe
>>>>
>>>>
>>>>
>>>> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>>>>>>   this JVM didn't last long ;)
>>>>>
>>>>> If only we got a penny for every JVM crash... :)
>>>>>
>>>>> Uwe -- success at reproducing?
>>>>>
>>>>> D.
>>>>>
>>>>> ________________________________
>>>>>
>>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>>>> additional commands, e-mail: dev-help@lucene.apache.org
>>>>>
>>>> --
>>>> Uwe Schindler
>>>> H.-H.-Meier-Allee 63, 28213 Bremen
>>>> http://www.thetaphi.de
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>>> additional commands, e-mail: dev-help@lucene.apache.org

-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

In addition, this is a GC bug, so it happens in the GC threads - which explains why it does not easy reproduce:
http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/artifact/lucene/build/core/test/J0/hs_err_pid11654.log

This one happens in ConcMarkSweepGC!

Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC

Current thread (0xf6a14000):  GCTaskThread [stack: 0xf447f000,0xf4500000] [id=11664]

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR), si_addr=0x00000004

Stack: [0xf447f000,0xf4500000],  sp=0xf44fef90,  free space=511k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V  [libjvm.so+0x4475f1]  ObjArrayKlass::oop_oop_iterate_nv(oopDesc*, ParScanWithoutBarrierClosure*)+0x61
V  [libjvm.so+0x47326f]  ParScanThreadState::trim_queues(int)+0xdf
V  [libjvm.so+0x473762]  ParEvacuateFollowersClosure::do_void()+0x22
V  [libjvm.so+0x4736ed]  ParNewGenTask::work(unsigned int)+0x12d V  [libjvm.so+0x58b6d9]  GangWorker::loop()+0xe9 V  [libjvm.so+0x58b2c8]  GangWorker::run()+0x18 V  [libjvm.so+0x461689]  java_start(Thread*)+0x119 C  [libpthread.so.0+0x6d4c]  start_thread+0xcc

I think the only thing we can do is to open a bug report!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Monday, November 25, 2013 12:18 PM
> To: 'dev@lucene.apache.org'
> Subject: RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
> # 8349 - Failure!
> 
> Same here. I also ran the whole test suite, does not reproduce!
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> 
> > -----Original Message-----
> > From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf
> > Of Dawid Weiss
> > Sent: Monday, November 25, 2013 11:04 AM
> > To: dev@lucene.apache.org
> > Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116)
> > - Build # 8349 - Failure!
> >
> > I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
> >
> > Dawid
> >
> > On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de>
> wrote:
> > > Not yet tried!
> > > I am not at home at the moment.
> > >
> > > Uwe
> > >
> > >
> > >
> > > Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
> > >>>
> > >>>  this JVM didn't last long ;)
> > >>
> > >>
> > >> If only we got a penny for every JVM crash... :)
> > >>
> > >> Uwe -- success at reproducing?
> > >>
> > >> D.
> > >>
> > >> ________________________________
> > >>
> > >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > >> additional commands, e-mail: dev-help@lucene.apache.org
> > >>
> > >
> > > --
> > > Uwe Schindler
> > > H.-H.-Meier-Allee 63, 28213 Bremen
> > > http://www.thetaphi.de
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Same here. I also ran the whole test suite, does not reproduce!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: dawid.weiss@gmail.com [mailto:dawid.weiss@gmail.com] On Behalf
> Of Dawid Weiss
> Sent: Monday, November 25, 2013 11:04 AM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
> # 8349 - Failure!
> 
> I can't reproduce this. I've ran the same (?) test args 20 times, no luck.
> 
> Dawid
> 
> On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> > Not yet tried!
> > I am not at home at the moment.
> >
> > Uwe
> >
> >
> >
> > Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
> >>>
> >>>  this JVM didn't last long ;)
> >>
> >>
> >> If only we got a penny for every JVM crash... :)
> >>
> >> Uwe -- success at reproducing?
> >>
> >> D.
> >>
> >> ________________________________
> >>
> >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> >> additional commands, e-mail: dev-help@lucene.apache.org
> >>
> >
> > --
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, 28213 Bremen
> > http://www.thetaphi.de
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
I can't reproduce this. I've ran the same (?) test args 20 times, no luck.

Dawid

On Sun, Nov 24, 2013 at 8:43 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Not yet tried!
> I am not at home at the moment.
>
> Uwe
>
>
>
> Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>>>
>>>  this JVM didn't last long ;)
>>
>>
>> If only we got a penny for every JVM crash... :)
>>
>> Uwe -- success at reproducing?
>>
>> D.
>>
>> ________________________________
>>
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
> --
> Uwe Schindler
> H.-H.-Meier-Allee 63, 28213 Bremen
> http://www.thetaphi.de

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Not yet tried! 
I am not at home at the moment. 

Uwe



Dawid Weiss <da...@cs.put.poznan.pl> schrieb:
>> this JVM didn't last long ;)
>
>If only we got a penny for every JVM crash... :)
>
>Uwe -- success at reproducing?
>
>D.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>For additional commands, e-mail: dev-help@lucene.apache.org

--
Uwe Schindler
H.-H.-Meier-Allee 63, 28213 Bremen
http://www.thetaphi.de

Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
> this JVM didn't last long ;)

If only we got a penny for every JVM crash... :)

Uwe -- success at reproducing?

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Simon Willnauer <si...@gmail.com>.
this JVM didn't last long ;)

On Sun, Nov 24, 2013 at 3:51 PM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Oh oh, let's try to reproduce!
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Robert Muir [mailto:rcmuir@gmail.com]
>> Sent: Sunday, November 24, 2013 2:36 PM
>> To: dev@lucene.apache.org
>> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
>> # 8349 - Failure!
>>
>> thats a new crash!
>>
>> On Sun, Nov 24, 2013 at 8:31 AM, Policeman Jenkins Server
>> <je...@thetaphi.de> wrote:
>> > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/
>> > Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>> >
>> > All tests passed
>> >
>> > Build Log:
>> > [...truncated 1126 lines...]
>> >    [junit4] JVM J0: stdout was not empty, see:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.sysout
>> >    [junit4] >>> JVM J0: stdout (verbatim) ----
>> >    [junit4] #
>> >    [junit4] # A fatal error has been detected by the Java Runtime
>> Environment:
>> >    [junit4] #
>> >    [junit4] #  SIGSEGV (0xb) at pc=0xf6fc55f1, pid=11654, tid=4098882368
>> >    [junit4] #
>> >    [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0-b116) (build
>> 1.8.0-ea-b116)
>> >    [junit4] # Java VM: Java HotSpot(TM) Client VM (25.0-b58 mixed mode
>> linux-x86 )
>> >    [junit4] # Problematic frame:
>> >    [junit4] # V  [libjvm.so+0x4475f1]
>> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
>> ParScanWithoutBarrierClosure*)+0x61
>> >    [junit4] #
>> >    [junit4] # Failed to write core dump. Core dumps have been disabled. To
>> enable core dumping, try "ulimit -c unlimited" before starting Java again
>> >    [junit4] #
>> >    [junit4] # An error report file with more information is saved as:
>> >    [junit4] # /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/test/J0/hs_err_pid11654.log
>> >    [junit4] #
>> >    [junit4] # If you would like to submit a bug report, please visit:
>> >    [junit4] #   http://bugreport.sun.com/bugreport/crash.jsp
>> >    [junit4] #
>> >    [junit4] <<< JVM J0: EOF ----
>> >
>> > [...truncated 454 lines...]
>> >    [junit4] ERROR: JVM J0 ended with an exception, command line:
>> /var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/jre/bin/java -client -
>> XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -
>> XX:HeapDumpPath=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/heapdumps -Dtests.prefix=tests -Dtests.seed=3861E4A6DB805158 -
>> Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -
>> Dtests.codec=random -Dtests.postingsformat=random -
>> Dtests.docvaluesformat=random -Dtests.locale=random -
>> Dtests.timezone=random -Dtests.directory=random -
>> Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -
>> Dtests.cleanthreads=perMethod -
>> Djava.util.logging.config.file=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -
>> Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -
>> Dtests.multiplier=3 -DtempDir=. -Djava.io.tmpdir=. -
>> Djunit4.tempDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/test/temp -
>> Dclover.db.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/clover/db -
>> Djava.security.manager=org.apache.lucene.util.TestSecurityManager -
>> Djava.security.policy=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -
>> Djetty.testMode=1 -Djetty.insecurerandom=1 -
>> Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -
>> Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -
>> Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-
>> framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucen
>> e-Solr-4.x-Linux/lucene/test-framework/lib/junit-
>> 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-
>> framework/lib/randomizedtesting-runner-
>> 2.0.13.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-
>> Solr-4.x-
>> Linux/lucene/build/core/classes/test:/var/lib/jenkins/tools/hudson.tasks.An
>> t_AntInstallation/ANT_1.8.2/lib/ant-
>> launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
>> 2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
>> lib/ant-
>> jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib
>> /ant-
>> swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
>> /lib/ant-apache-
>> oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
>> b/ant-
>> jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
>> b/ant-apache-
>> xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
>> 2/lib/ant-
>> javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
>> 8.2/lib/ant-apache-
>> resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
>> 8.2/lib/ant-
>> testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
>> 2/lib/ant-commons-
>> logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
>> 2/lib/ant-apache-
>> log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
>> lib/ant-
>> junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
>> lib/ant-
>> jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
>> b/ant-commons-
>> net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
>> b/ant-apache-
>> bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
>> b/ant-
>> jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
>> 8.2/lib/ant-
>> netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
>> 8.2/lib/ant-apache-
>> regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
>> 2/lib/ant-
>> junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
>> /lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8
>> .2/lib/ant-apache-
>> bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/l
>> ib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-
>> b116/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomized
>> testing/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -
>> ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -
>> eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.events
>> @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
>> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.suites
>> >    [junit4] ERROR: JVM J0 ended with an exception: Forked process
>> returned with error code: 134. Very likely a JVM crash.  Process output piped
>> in logs above.
>> >    [junit4]     at
>> com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
>> >    [junit4]     at
>> com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
>> >    [junit4]     at
>> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
>> >    [junit4]     at
>> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
>> >    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> >    [junit4]     at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
>> a:1142)
>> >    [junit4]     at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
>> va:617)
>> >    [junit4]     at java.lang.Thread.run(Thread.java:744)
>> >
>> > BUILD FAILED
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The
>> following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The
>> following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The
>> following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37:
>> The following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49:
>> The following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-
>> build.xml:1279: The following error occurred while executing this line:
>> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-
>> build.xml:912: At least one slave process threw an exception, first: Forked
>> process returned with error code: 134. Very likely a JVM crash.  Process
>> output piped in logs above.
>> >
>> > Total time: 14 minutes 16 seconds
>> > Build step 'Invoke Ant' marked build as failure Description set: Java:
>> > 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC Archiving
>> > artifacts Recording test results Email was triggered for: Failure
>> > Sending email for trigger: Failure
>> >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Oh oh, let's try to reproduce!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Robert Muir [mailto:rcmuir@gmail.com]
> Sent: Sunday, November 24, 2013 2:36 PM
> To: dev@lucene.apache.org
> Subject: Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build
> # 8349 - Failure!
> 
> thats a new crash!
> 
> On Sun, Nov 24, 2013 at 8:31 AM, Policeman Jenkins Server
> <je...@thetaphi.de> wrote:
> > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/
> > Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
> >
> > All tests passed
> >
> > Build Log:
> > [...truncated 1126 lines...]
> >    [junit4] JVM J0: stdout was not empty, see:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.sysout
> >    [junit4] >>> JVM J0: stdout (verbatim) ----
> >    [junit4] #
> >    [junit4] # A fatal error has been detected by the Java Runtime
> Environment:
> >    [junit4] #
> >    [junit4] #  SIGSEGV (0xb) at pc=0xf6fc55f1, pid=11654, tid=4098882368
> >    [junit4] #
> >    [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0-b116) (build
> 1.8.0-ea-b116)
> >    [junit4] # Java VM: Java HotSpot(TM) Client VM (25.0-b58 mixed mode
> linux-x86 )
> >    [junit4] # Problematic frame:
> >    [junit4] # V  [libjvm.so+0x4475f1]
> ObjArrayKlass::oop_oop_iterate_nv(oopDesc*,
> ParScanWithoutBarrierClosure*)+0x61
> >    [junit4] #
> >    [junit4] # Failed to write core dump. Core dumps have been disabled. To
> enable core dumping, try "ulimit -c unlimited" before starting Java again
> >    [junit4] #
> >    [junit4] # An error report file with more information is saved as:
> >    [junit4] # /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/test/J0/hs_err_pid11654.log
> >    [junit4] #
> >    [junit4] # If you would like to submit a bug report, please visit:
> >    [junit4] #   http://bugreport.sun.com/bugreport/crash.jsp
> >    [junit4] #
> >    [junit4] <<< JVM J0: EOF ----
> >
> > [...truncated 454 lines...]
> >    [junit4] ERROR: JVM J0 ended with an exception, command line:
> /var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/jre/bin/java -client -
> XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -
> XX:HeapDumpPath=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/heapdumps -Dtests.prefix=tests -Dtests.seed=3861E4A6DB805158 -
> Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -
> Dtests.codec=random -Dtests.postingsformat=random -
> Dtests.docvaluesformat=random -Dtests.locale=random -
> Dtests.timezone=random -Dtests.directory=random -
> Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -
> Dtests.cleanthreads=perMethod -
> Djava.util.logging.config.file=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -
> Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -
> Dtests.multiplier=3 -DtempDir=. -Djava.io.tmpdir=. -
> Djunit4.tempDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/test/temp -
> Dclover.db.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/clover/db -
> Djava.security.manager=org.apache.lucene.util.TestSecurityManager -
> Djava.security.policy=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -
> Djetty.testMode=1 -Djetty.insecurerandom=1 -
> Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -
> Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -
> Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-
> framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucen
> e-Solr-4.x-Linux/lucene/test-framework/lib/junit-
> 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-
> framework/lib/randomizedtesting-runner-
> 2.0.13.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-
> Solr-4.x-
> Linux/lucene/build/core/classes/test:/var/lib/jenkins/tools/hudson.tasks.An
> t_AntInstallation/ANT_1.8.2/lib/ant-
> launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
> 2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
> lib/ant-
> jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib
> /ant-
> swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> /lib/ant-apache-
> oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
> b/ant-
> jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
> b/ant-apache-
> xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
> 2/lib/ant-
> javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
> 8.2/lib/ant-apache-
> resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
> 8.2/lib/ant-
> testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
> 2/lib/ant-commons-
> logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
> 2/lib/ant-apache-
> log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
> lib/ant-
> junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/
> lib/ant-
> jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
> b/ant-commons-
> net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
> b/ant-apache-
> bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/li
> b/ant-
> jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
> 8.2/lib/ant-
> netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.
> 8.2/lib/ant-apache-
> regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.
> 2/lib/ant-
> junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
> /lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8
> .2/lib/ant-apache-
> bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/l
> ib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-
> b116/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomized
> testing/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -
> ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -
> eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.events
> @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
> Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.suites
> >    [junit4] ERROR: JVM J0 ended with an exception: Forked process
> returned with error code: 134. Very likely a JVM crash.  Process output piped
> in logs above.
> >    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
> >    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
> >    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
> >    [junit4]     at
> com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
> >    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> >    [junit4]     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.jav
> a:1142)
> >    [junit4]     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.ja
> va:617)
> >    [junit4]     at java.lang.Thread.run(Thread.java:744)
> >
> > BUILD FAILED
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The
> following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The
> following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The
> following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37:
> The following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49:
> The following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-
> build.xml:1279: The following error occurred while executing this line:
> > /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-
> build.xml:912: At least one slave process threw an exception, first: Forked
> process returned with error code: 134. Very likely a JVM crash.  Process
> output piped in logs above.
> >
> > Total time: 14 minutes 16 seconds
> > Build step 'Invoke Ant' marked build as failure Description set: Java:
> > 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC Archiving
> > artifacts Recording test results Email was triggered for: Failure
> > Sending email for trigger: Failure
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b116) - Build # 8349 - Failure!

Posted by Robert Muir <rc...@gmail.com>.
thats a new crash!

On Sun, Nov 24, 2013 at 8:31 AM, Policeman Jenkins Server
<je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8349/
> Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
>
> All tests passed
>
> Build Log:
> [...truncated 1126 lines...]
>    [junit4] JVM J0: stdout was not empty, see: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.sysout
>    [junit4] >>> JVM J0: stdout (verbatim) ----
>    [junit4] #
>    [junit4] # A fatal error has been detected by the Java Runtime Environment:
>    [junit4] #
>    [junit4] #  SIGSEGV (0xb) at pc=0xf6fc55f1, pid=11654, tid=4098882368
>    [junit4] #
>    [junit4] # JRE version: Java(TM) SE Runtime Environment (8.0-b116) (build 1.8.0-ea-b116)
>    [junit4] # Java VM: Java HotSpot(TM) Client VM (25.0-b58 mixed mode linux-x86 )
>    [junit4] # Problematic frame:
>    [junit4] # V  [libjvm.so+0x4475f1]  ObjArrayKlass::oop_oop_iterate_nv(oopDesc*, ParScanWithoutBarrierClosure*)+0x61
>    [junit4] #
>    [junit4] # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
>    [junit4] #
>    [junit4] # An error report file with more information is saved as:
>    [junit4] # /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/J0/hs_err_pid11654.log
>    [junit4] #
>    [junit4] # If you would like to submit a bug report, please visit:
>    [junit4] #   http://bugreport.sun.com/bugreport/crash.jsp
>    [junit4] #
>    [junit4] <<< JVM J0: EOF ----
>
> [...truncated 454 lines...]
>    [junit4] ERROR: JVM J0 ended with an exception, command line: /var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/jre/bin/java -client -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/heapdumps -Dtests.prefix=tests -Dtests.seed=3861E4A6DB805158 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp -Dclover.db.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.disableHdfs=true -Dfile.encoding=UTF-8 -classpath /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.0.13.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/test:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b116/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.events @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/test/temp/junit4-J0-20131124_131747_681.suites
>    [junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
>    [junit4]     at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
>    [junit4]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>    [junit4]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>    [junit4]     at java.lang.Thread.run(Thread.java:744)
>
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1279: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
>
> Total time: 14 minutes 16 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 32bit/jdk1.8.0-ea-b116 -client -XX:+UseConcMarkSweepGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org