You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/07/29 18:38:52 UTC
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4784 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4784/
3 tests failed.
REGRESSION: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
IOException occured when talking to server at: https://127.0.0.1:15165
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: https://127.0.0.1:15165
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:563)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.request.CollectionAdminRequest.process(CollectionAdminRequest.java:367)
at org.apache.solr.client.solrj.request.CollectionAdminRequest.requestStatus(CollectionAdminRequest.java:516)
at org.apache.solr.cloud.MultiThreadedOCPTest.getRequestState(MultiThreadedOCPTest.java:249)
at org.apache.solr.cloud.MultiThreadedOCPTest.getRequestState(MultiThreadedOCPTest.java:245)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:120)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:196)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.read(InputRecord.java:480)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:449)
... 49 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest: 1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] at java.io.FilterOutputStream.write(FilterOutputStream.java:77) at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:134) at java.io.FilterOutputStream.write(FilterOutputStream.java:125) at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:128) at java.io.PrintStream.write(PrintStream.java:480) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295) at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141) at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229) at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59) at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324) at org.apache.log4j.WriterAppender.append(WriterAppender.java:162) at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251) at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66) at org.apache.log4j.Category.callAppenders(Category.java:206) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest:
1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
at java.io.FilterOutputStream.write(FilterOutputStream.java:77)
at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:134)
at java.io.FilterOutputStream.write(FilterOutputStream.java:125)
at org.apache.lucene.util.TestRuleLimitSysouts$DelegateStream.write(TestRuleLimitSysouts.java:128)
at java.io.PrintStream.write(PrintStream.java:480)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
at __randomizedtesting.SeedInfo.seed([1FA00104FEC5EDFC]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.MultiThreadedOCPTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest] at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251) at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66) at org.apache.log4j.Category.callAppenders(Category.java:206) at org.apache.log4j.Category.forcedLog(Category.java:391) at org.apache.log4j.Category.log(Category.java:856) at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
at org.apache.log4j.Category.callAppenders(Category.java:206)
at org.apache.log4j.Category.forcedLog(Category.java:391)
at org.apache.log4j.Category.log(Category.java:856)
at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
at __randomizedtesting.SeedInfo.seed([1FA00104FEC5EDFC]:0)
Build Log:
[...truncated 11371 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/init-core-data-001
[junit4] 2> 1580147 T3365 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 1580148 T3365 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 1580153 T3365 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1580154 T3365 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1580155 T3366 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1580254 T3365 oasc.ZkTestServer.run start zk server on port:15157
[junit4] 2> 1580255 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1580259 T3372 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@370db281 name:ZooKeeperConnection Watcher:127.0.0.1:15157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1580259 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1580259 T3365 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1580263 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1580264 T3374 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@261f3097 name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1580264 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1580265 T3365 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1580267 T3365 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1580269 T3365 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1580271 T3365 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1580273 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1580273 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1580277 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1580277 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1580380 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1580380 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1580383 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1580383 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1580386 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1580386 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1580389 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1580389 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1580391 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1580392 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1580394 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1580395 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1580400 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1580401 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1580404 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1580405 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1580407 T3365 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1580408 T3365 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1580411 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1580412 T3376 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5701bd00 name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1580413 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1581756 T3365 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1581763 T3365 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1581765 T3365 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:15161
[junit4] 2> 1581768 T3365 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1581768 T3365 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1581769 T3365 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002
[junit4] 2> 1581769 T3365 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/'
[junit4] 2> 1581817 T3365 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/solr.xml
[junit4] 2> 1581896 T3365 oasc.CoreContainer.<init> New CoreContainer 845460342
[junit4] 2> 1581896 T3365 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/]
[junit4] 2> 1581898 T3365 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1581898 T3365 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1581899 T3365 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1581899 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1581899 T3365 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1581900 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1581900 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1581900 T3365 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1581901 T3365 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1581904 T3365 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1581906 T3365 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1581907 T3365 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1581907 T3365 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1581907 T3365 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15157/solr
[junit4] 2> 1581908 T3365 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1581909 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1581910 T3387 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fdb8e0e name:ZooKeeperConnection Watcher:127.0.0.1:15157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1581911 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1581913 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1581915 T3389 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4eaba0b4 name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1581915 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1581917 T3365 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1581919 T3365 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1581922 T3365 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1581924 T3365 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1581927 T3365 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1581929 T3365 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1581931 T3365 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15161_
[junit4] 2> 1581932 T3365 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15161_
[junit4] 2> 1581934 T3365 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1581936 T3365 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1581939 T3365 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:15161_
[junit4] 2> 1581939 T3365 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1581941 T3365 oasc.Overseer.start Overseer (id=92186272179224580-127.0.0.1:15161_-n_0000000000) starting
[junit4] 2> 1581943 T3365 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1581951 T3391 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1581951 T3365 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1581953 T3365 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1581954 T3365 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1581957 T3390 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1581962 T3392 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1581963 T3392 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1581964 T3392 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1581964 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1581965 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1581966 T3390 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:15161",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15161_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1581966 T3390 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1581966 T3390 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1581968 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1581969 T3393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1582964 T3392 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1582965 T3392 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1582966 T3392 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1582966 T3392 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1582967 T3392 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1582967 T3392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/collection1/'
[junit4] 2> 1582969 T3392 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 1582970 T3392 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 1582971 T3392 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 1583037 T3392 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1583085 T3392 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1583186 T3392 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1583198 T3392 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1583623 T3392 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1583637 T3392 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1583640 T3392 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1583654 T3392 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1583658 T3392 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1583662 T3392 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1583663 T3392 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1583664 T3392 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1583664 T3392 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1583665 T3392 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1583665 T3392 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1583666 T3392 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1583666 T3392 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1583666 T3392 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-001/control/data/
[junit4] 2> 1583667 T3392 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5443c95b
[junit4] 2> 1583668 T3392 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-001/control/data
[junit4] 2> 1583668 T3392 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-001/control/data/index/
[junit4] 2> 1583668 T3392 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1583669 T3392 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-001/control/data/index
[junit4] 2> 1583669 T3392 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1583670 T3392 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@32a505a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d9bc244),segFN=segments_1,generation=1}
[junit4] 2> 1583670 T3392 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1583675 T3392 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1583676 T3392 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1583676 T3392 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1583677 T3392 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1583677 T3392 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1583677 T3392 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1583678 T3392 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1583678 T3392 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1583678 T3392 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1583679 T3392 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1583680 T3392 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1583680 T3392 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1583680 T3392 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1583681 T3392 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1583681 T3392 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1583682 T3392 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1583682 T3392 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1583701 T3392 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1583707 T3392 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1583707 T3392 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1583708 T3392 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1583708 T3392 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@32a505a6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d9bc244),segFN=segments_1,generation=1}
[junit4] 2> 1583709 T3392 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1583709 T3392 oass.SolrIndexSearcher.<init> Opening Searcher@40204801[collection1] main
[junit4] 2> 1583709 T3392 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1583710 T3392 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1583710 T3392 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1583711 T3392 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1583711 T3392 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1583712 T3392 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1583712 T3392 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1583712 T3392 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1583713 T3392 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 1583715 T3392 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 1583715 T3392 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1583716 T3392 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1583719 T3394 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40204801[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1583722 T3392 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1583723 T3397 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:15161 collection:control_collection shard:shard1
[junit4] 2> 1583723 T3365 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4] 2> 1583723 T3365 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1583724 T3397 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1583729 T3397 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1583729 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1583730 T3399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33e07804 name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1583731 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1583731 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583732 T3397 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1583732 T3397 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 1583732 T3365 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1583732 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> ASYNC NEW_CORE C642 name=collection1 org.apache.solr.core.SolrCore@5fe9f3c9 url=https://127.0.0.1:15161/collection1 node=127.0.0.1:15161_ C642_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:15161, core=collection1, node_name=127.0.0.1:15161_}
[junit4] 2> 1583732 T3397 C642 P15161 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:15161/collection1/
[junit4] 2> 1583733 T3397 C642 P15161 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1583733 T3397 C642 P15161 oasc.SyncStrategy.syncToMe https://127.0.0.1:15161/collection1/ has no replicas
[junit4] 2> 1583734 T3397 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:15161/collection1/ shard1
[junit4] 2> 1583734 T3397 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1583734 T3365 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1583735 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583735 T3393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1583736 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1583739 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583740 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1583742 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583843 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1583844 T3393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1583891 T3397 oasc.ZkController.register We are https://127.0.0.1:15161/collection1/ and leader is https://127.0.0.1:15161/collection1/
[junit4] 2> 1583892 T3397 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:15161
[junit4] 2> 1583892 T3397 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1583892 T3397 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1583892 T3397 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1583894 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583894 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583894 T3397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1583894 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583895 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1583896 T3390 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:15161",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15161_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1583898 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1583999 T3402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1583999 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1584818 T3365 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1584819 T3365 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1584823 T3365 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1584825 T3365 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:15165
[junit4] 2> 1584827 T3365 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1584827 T3365 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1584827 T3365 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003
[junit4] 2> 1584828 T3365 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/'
[junit4] 2> 1584860 T3365 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/solr.xml
[junit4] 2> 1584913 T3365 oasc.CoreContainer.<init> New CoreContainer 1779305860
[junit4] 2> 1584913 T3365 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/]
[junit4] 2> 1584914 T3365 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1584915 T3365 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1584915 T3365 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1584915 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1584915 T3365 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1584916 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1584916 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1584916 T3365 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1584916 T3365 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1584918 T3365 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1584920 T3365 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1584920 T3365 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1584921 T3365 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1584921 T3365 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15157/solr
[junit4] 2> 1584921 T3365 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1584922 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1584923 T3413 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4590e1a3 name:ZooKeeperConnection Watcher:127.0.0.1:15157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1584924 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1584926 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1584927 T3415 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@313589c name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1584928 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1584933 T3365 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1585936 T3365 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15165_
[junit4] 2> 1585937 T3365 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15165_
[junit4] 2> 1585948 T3417 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1585948 T3417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1585949 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1585949 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1585949 T3417 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1585949 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1585950 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1585951 T3390 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:15165",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15165_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1585951 T3390 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 1585952 T3390 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1585953 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1585954 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1585954 T3401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1585954 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1586950 T3417 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1586951 T3417 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1586951 T3417 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1586952 T3417 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1586953 T3417 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1586953 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/'
[junit4] 2> 1586954 T3417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 1586955 T3417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 1586956 T3417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 1587022 T3417 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1587070 T3417 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1587171 T3417 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1587182 T3417 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1587580 T3417 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1587593 T3417 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1587596 T3417 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1587610 T3417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1587615 T3417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1587619 T3417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1587620 T3417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1587621 T3417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1587621 T3417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1587622 T3417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1587622 T3417 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1587623 T3417 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1587623 T3417 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1587623 T3417 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/data/
[junit4] 2> 1587624 T3417 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5443c95b
[junit4] 2> 1587624 T3417 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/data
[junit4] 2> 1587625 T3417 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/data/index/
[junit4] 2> 1587625 T3417 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1587625 T3417 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-003/collection1/data/index
[junit4] 2> 1587626 T3417 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1587627 T3417 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4d879e18 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13ea1980),segFN=segments_1,generation=1}
[junit4] 2> 1587627 T3417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1587634 T3417 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1587634 T3417 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1587635 T3417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1587635 T3417 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1587635 T3417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1587635 T3417 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1587636 T3417 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1587637 T3417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1587637 T3417 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1587638 T3417 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1587638 T3417 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1587638 T3417 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1587639 T3417 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1587640 T3417 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1587640 T3417 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1587641 T3417 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1587641 T3417 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1587660 T3417 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1587665 T3417 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1587665 T3417 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1587666 T3417 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1587667 T3417 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@4d879e18 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13ea1980),segFN=segments_1,generation=1}
[junit4] 2> 1587667 T3417 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1587667 T3417 oass.SolrIndexSearcher.<init> Opening Searcher@108e54f1[collection1] main
[junit4] 2> 1587668 T3417 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1587669 T3417 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1587669 T3417 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1587669 T3417 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1587670 T3417 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1587670 T3417 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1587671 T3417 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1587671 T3417 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1587671 T3417 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1587672 T3417 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1587676 T3418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@108e54f1[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1587680 T3417 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1587681 T3421 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:15165 collection:collection1 shard:shard2
[junit4] 2> 1587681 T3365 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4] 2> 1587682 T3365 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1587682 T3421 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1587687 T3421 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1587688 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587689 T3421 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1587689 T3421 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C643 name=collection1 org.apache.solr.core.SolrCore@5a1fbf80 url=https://127.0.0.1:15165/collection1 node=127.0.0.1:15165_ C643_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:15165, core=collection1, node_name=127.0.0.1:15165_}
[junit4] 2> 1587689 T3421 C643 P15165 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:15165/collection1/
[junit4] 2> 1587690 T3421 C643 P15165 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1587690 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1587690 T3421 C643 P15165 oasc.SyncStrategy.syncToMe https://127.0.0.1:15165/collection1/ has no replicas
[junit4] 2> 1587690 T3421 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:15165/collection1/ shard2
[junit4] 2> 1587691 T3421 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1587692 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587693 T3401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587693 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587693 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587696 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1587698 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587799 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587799 T3401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587799 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587848 T3421 oasc.ZkController.register We are https://127.0.0.1:15165/collection1/ and leader is https://127.0.0.1:15165/collection1/
[junit4] 2> 1587849 T3421 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:15165
[junit4] 2> 1587849 T3421 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1587849 T3421 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1587849 T3421 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1587851 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587851 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587851 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587851 T3421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1587852 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1587853 T3390 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:15165",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15165_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1587855 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1587957 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587957 T3393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1587957 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1588771 T3365 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1588772 T3365 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1588777 T3365 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1588778 T3365 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:15168
[junit4] 2> 1588780 T3365 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1588780 T3365 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1588781 T3365 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004
[junit4] 2> 1588781 T3365 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/'
[junit4] 2> 1588813 T3365 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/solr.xml
[junit4] 2> 1588865 T3365 oasc.CoreContainer.<init> New CoreContainer 1884151070
[junit4] 2> 1588865 T3365 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/]
[junit4] 2> 1588866 T3365 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1588867 T3365 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1588867 T3365 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1588867 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1588867 T3365 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1588868 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1588868 T3365 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1588868 T3365 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1588868 T3365 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1588870 T3365 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1588872 T3365 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1588872 T3365 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1588873 T3365 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1588873 T3365 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15157/solr
[junit4] 2> 1588873 T3365 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1588874 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1588875 T3432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@372db784 name:ZooKeeperConnection Watcher:127.0.0.1:15157 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1588876 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1588878 T3365 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1588879 T3434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69c9444 name:ZooKeeperConnection Watcher:127.0.0.1:15157/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1588879 T3365 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1588885 T3365 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1589888 T3365 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15168_
[junit4] 2> 1589889 T3365 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15168_
[junit4] 2> 1589899 T3436 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1589900 T3436 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1589901 T3393 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1589901 T3401 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1589901 T3436 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1589901 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1589902 T3390 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1589903 T3390 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:15168",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:15168_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1589903 T3390 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1589903 T3390 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1589905 T3402 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1589906 T3402 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1589906 T3416 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1589906 T3435 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1589906 T3400 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1590901 T3436 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1590902 T3436 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1590903 T3436 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1590903 T3436 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1590904 T3436 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1590904 T3436 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/'
[junit4] 2> 1590906 T3436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 1590907 T3436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 1590908 T3436 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 1590986 T3436 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1591040 T3436 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1591142 T3436 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1591154 T3436 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1591597 T3436 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1591614 T3436 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1591617 T3436 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1591631 T3436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1591635 T3436 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1591640 T3436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1591640 T3436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1591642 T3436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1591642 T3436 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1591643 T3436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1591643 T3436 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1591643 T3436 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1591644 T3436 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1591644 T3436 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/data/
[junit4] 2> 1591644 T3436 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5443c95b
[junit4] 2> 1591645 T3436 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/data
[junit4] 2> 1591646 T3436 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/data/index/
[junit4] 2> 1591646 T3436 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1591646 T3436 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-1FA00104FEC5EDFC-001/tempDir-004/collection1/data/index
[junit4] 2> 1591647 T3436 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=16, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1591648 T3436 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6f379f64 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@69c29b04),segFN=segments_1,generation=1}
[junit4] 2> 1591648 T3436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1591655 T3436 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1591655 T3436 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1591656 T3436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1591656 T3436 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1591656 T3436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1591656 T3436 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1591657 T3436 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1591657 T3436 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1591658 T3436 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1591658 T3436 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1591659 T3436 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1591659 T3436 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1591660 T3436 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1591660 T3436 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1591661 T3436 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1591661 T3436 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1591662 T3436 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1591689 T3436 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1591695 T3436 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1591696 T3436 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1591697 T3436 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1591697 T3436 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@6f379f64 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@69c29b04),segFN=segments_1,generation=1}
[junit4] 2> 1591698 T3436 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1591698 T3436 oass.SolrIndexSearcher.<init> Opening Searcher@7578ea36[collection1] main
[junit4] 2> 1591698 T3436 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1591699 T3436 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1591699 T3436 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1591700 T3436 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1591700 T3436 oasr.RestManager.init Initializing RestManager with init
[...truncated too long message...]
, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 86878]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: All tests run in this JVM: [TestDistributedMissingSort, UnloadDistributedZkTest, TestSearchPerf, SolrCoreCheckLockOnStartupTest, SyncSliceTest, NoCacheHeaderTest, PathHierarchyTokenizerFactoryTest, TestManagedSchema, TimeZoneUtilsTest, TestFaceting, RequiredFieldsTest, OverseerRolesTest, ClusterStateUpdateTest, TestSurroundQueryParser, PeerSyncTest, FieldMutatingUpdateProcessorTest, ShardRoutingTest, SpellCheckCollatorTest, ExternalFileFieldSortTest, TestArbitraryIndexDir, TestSolrXmlPersistor, MoreLikeThisHandlerTest, QueryFacetTest, UpdateParamsTest, TestRecoveryHdfs, NotRequiredUniqueKeyTest, MBeansHandlerTest, TestHighFrequencyDictionaryFactory, TestCloudSchemaless, TestRandomDVFaceting, CollectionsAPIAsyncDistributedZkTest, JsonLoaderTest, DocumentBuilderTest, DocExpirationUpdateProcessorFactoryTest, TestNRTOpen, TestSerializedLuceneMatchVersion, OverseerCollectionProcessorTest, CoreAdminCreateDiscoverTest, TestWordDelimiterFilterFactory, TestDefaultSimilarityFactory, TestUniqueKeyFieldResource, TermsComponentTest, RegexBoostProcessorTest, TestRandomMergePolicy, OpenExchangeRatesOrgProviderTest, AnalyticsQueryTest, SolrTestCaseJ4Test, BadIndexSchemaTest, ZkNodePropsTest, TestCollapseQParserPlugin, SolrCmdDistributorTest, TestShortCircuitedRequests, SignatureUpdateProcessorFactoryTest, TestFastLRUCache, TestDistributedSearch, HdfsDirectoryTest, TestFastWriter, ChaosMonkeyNothingIsSafeTest, ShowFileRequestHandlerTest, StatelessScriptUpdateProcessorFactoryTest, PreAnalyzedFieldTest, TestFoldingMultitermQuery, TestFastOutputStream, MinimalSchemaTest, JSONWriterTest, DistributedQueryComponentOptimizationTest, MultiThreadedOCPTest]
[junit4] 2> 1878964 T3389 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@185eedd6 rejected from java.util.concurrent.ThreadPoolExecutor@7ad09a4d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 86878]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.seed=1FA00104FEC5EDFC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=Atlantic/Reykjavik -Dtests.file.encoding=ISO-8859-1
[junit4] 2> 1878965 T3389 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@27590a1f rejected from java.util.concurrent.ThreadPoolExecutor@7ad09a4d[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 86878]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] ERROR 0.00s J0 | MultiThreadedOCPTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.MultiThreadedOCPTest:
[junit4] > 1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > at java.io.FilterOutputStream.write(FilterOutputStream.java:77)
[junit4] > at java.io.FilterOutputStream.write(FilterOutputStream.java:125)
[junit4] > at java.io.PrintStream.write(PrintStream.java:480)
[junit4] > at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
[junit4] > at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
[junit4] > at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
[junit4] > at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
[junit4] > at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
[junit4] > at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:59)
[junit4] > at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:324)
[junit4] > at org.apache.log4j.WriterAppender.append(WriterAppender.java:162)
[junit4] > at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
[junit4] > at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
[junit4] > at org.apache.log4j.Category.callAppenders(Category.java:206)
[junit4] > at org.apache.log4j.Category.forcedLog(Category.java:391)
[junit4] > at org.apache.log4j.Category.log(Category.java:856)
[junit4] > at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] > at __randomizedtesting.SeedInfo.seed([1FA00104FEC5EDFC]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3389, name=TEST-MultiThreadedOCPTest.testDistribSearch-seed#[1FA00104FEC5EDFC]-EventThread, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:251)
[junit4] > at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:66)
[junit4] > at org.apache.log4j.Category.callAppenders(Category.java:206)
[junit4] > at org.apache.log4j.Category.forcedLog(Category.java:391)
[junit4] > at org.apache.log4j.Category.log(Category.java:856)
[junit4] > at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:575)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:524)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] > at __randomizedtesting.SeedInfo.seed([1FA00104FEC5EDFC]:0)
[junit4] Completed on J0 in 298.85s, 1 test, 3 errors <<< FAILURES!
[...truncated 748 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 421 suites, 1753 tests, 2 suite-level errors, 1 error, 1026 ignored (13 assumptions)
Total time: 78 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 354983594 bytes
Compression is 0.0%
Took 57 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4792 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4792/
1 tests failed.
FAILED: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest.testExampleConfig
Error Message:
Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform. at java.lang.UNIXProcess$1.run(UNIXProcess.java:111) at java.lang.UNIXProcess$1.run(UNIXProcess.java:93) at java.security.AccessController.doPrivileged(Native Method) at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) at java.lang.Runtime.exec(Runtime.java:617) at java.lang.Runtime.exec(Runtime.java:450) at java.lang.Runtime.exec(Runtime.java:347) at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220) at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176) at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) ,code=500}</title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /solr/admin/info/system. Reason: <pre> {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform. at java.lang.UNIXProcess$1.run(UNIXProcess.java:111) at java.lang.UNIXProcess$1.run(UNIXProcess.java:93) at java.security.AccessController.doPrivileged(Native Method) at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91) at java.lang.ProcessImpl.start(ProcessImpl.java:130) at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022) at java.lang.Runtime.exec(Runtime.java:617) at java.lang.Runtime.exec(Runtime.java:450) at java.lang.Runtime.exec(Runtime.java:347) at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220) at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176) at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) ,code=500}</pre></p> <hr /><i><small>Powered by Jetty://</small></i> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
,code=500}</title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /solr/admin/info/system. Reason:
<pre> {msg=posix_spawn is not a supported process launch mechanism on this platform.,trace=java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:732)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:137)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
,code=500}</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([25EEF9A8D20C97FB:92C3ED30150B7122]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:513)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.client.solrj.SolrExampleTests.testExampleConfig(SolrExampleTests.java:220)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12578 lines...]
[junit4] Suite: org.apache.solr.client.solrj.embedded.SolrExampleStreamingTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/init-core-data-001
[junit4] 2> 161427 T6286 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 161428 T6286 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 161429 T6286 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 161434 T6286 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:59691
[junit4] 2> 161434 T6286 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 161435 T6286 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 161435 T6286 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
[junit4] 2> 161436 T6286 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/'
[junit4] 2> 161477 T6286 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/solr.xml
[junit4] 2> 161547 T6286 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
[junit4] 2> 161547 T6286 oasc.CoreContainer.<init> New CoreContainer 761401314
[junit4] 2> 161548 T6286 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/]
[junit4] 2> 161549 T6286 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 161549 T6286 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 161550 T6286 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 161550 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 161551 T6286 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 161552 T6286 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 161552 T6286 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 161553 T6286 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
[junit4] 2> 161553 T6286 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 161554 T6286 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 161554 T6286 oasc.CoreContainer.load Host Name:
[junit4] 2> 161558 T6286 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr
[junit4] 2> 161560 T6286 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/
[junit4] 2> 161572 T6286 oasc.CorePropertiesLocator.discover Found 1 core definitions
[junit4] 2> 161573 T6296 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/'
[junit4] 2> 161624 T6296 oasc.SolrConfig.initLibs Adding specified lib dirs to ClassLoader
[junit4] 2> 161625 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/extraction/lib).
[junit4] 2> 161626 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
[junit4] 2> 161626 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/clustering/lib).
[junit4] 2> 161627 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
[junit4] 2> 161628 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/langid/lib).
[junit4] 2> 161628 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
[junit4] 2> 161629 T6296 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../contrib/velocity/lib).
[junit4] 2> 161629 T6296 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/../../../dist).
[junit4] 2> 161693 T6296 oasu.SolrIndexConfig.<init> IndexWriter infoStream solr logging is enabled
[junit4] 2> 161698 T6296 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 161791 T6296 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 161792 T6296 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 161840 T6296 oass.IndexSchema.readSchema [collection1] Schema name=example
[junit4] 2> 162452 T6296 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 162454 T6296 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
[junit4] 2> 162454 T6296 oasr.RestManager$Registry.registerManagedResource Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
[junit4] 2> 162476 T6296 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 162482 T6296 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 162513 T6296 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/
[junit4] 2> 162513 T6296 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 162514 T6296 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/
[junit4] 2> 162514 T6296 oasc.JmxMonitoredMap.<init> No JMX servers found, not exposing Solr information with JMX.
[junit4] 2> 162514 T6296 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for newSearcher: org.apache.solr.core.QuerySenderListener{queries=[]}
[junit4] 2> 162514 T6296 oasc.SolrCore.initListeners [collection1] Added SolrEventListener for firstSearcher: org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}
[junit4] 2> 162515 T6296 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001
[junit4] 2> 162515 T6296 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index/
[junit4] 2> 162515 T6296 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index' doesn't exist. Creating new index...
[junit4] 2> 162516 T6296 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index
[junit4] 2> 162517 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: current segments file is "null"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@44dd1ac2
[junit4] 2> 162518 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
[junit4] 2> 162518 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
[junit4] 2> 162518 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: init: create=true
[junit4] 2> 162518 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]:
[junit4] 2> dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index)
[junit4] 2> index=
[junit4] 2> version=5.0-SNAPSHOT
[junit4] 2> matchVersion=LUCENE_5_0
[junit4] 2> analyzer=null
[junit4] 2> ramBufferSizeMB=100.0
[junit4] 2> maxBufferedDocs=-1
[junit4] 2> maxBufferedDeleteTerms=-1
[junit4] 2> mergedSegmentWarmer=null
[junit4] 2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
[junit4] 2> commit=null
[junit4] 2> openMode=CREATE
[junit4] 2> similarity=org.apache.lucene.search.similarities.DefaultSimilarity
[junit4] 2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2, mergeThreadPriority=-1
[junit4] 2> default WRITE_LOCK_TIMEOUT=1000
[junit4] 2> writeLockTimeout=1000
[junit4] 2> codec=Lucene49
[junit4] 2> infoStream=org.apache.solr.update.LoggingInfoStream
[junit4] 2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@1007d9e0
[junit4] 2> readerPooling=false
[junit4] 2> perThreadHardLimitMB=1945
[junit4] 2> useCompoundFile=false
[junit4] 2> checkIntegrityAtMerge=false
[junit4] 2> writer=org.apache.lucene.util.SetOnce@73710958
[junit4] 2>
[junit4] 2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: now flush at shutdown
[junit4] 2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: start flush: applyAllDeletes=true
[junit4] 2> 162519 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: index before flush
[junit4] 2> 162519 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
[junit4] 2> 162519 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162520 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
[junit4] 2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
[junit4] 2> 162520 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=null: minGen=9223372036854775807 packetCount=0
[junit4] 2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: waitForMerges
[junit4] 2> 162520 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: waitForMerges done
[junit4] 2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: start
[junit4] 2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: enter lock
[junit4] 2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: now prepare
[junit4] 2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: prepareCommit: flush
[junit4] 2> 162521 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: index before flush
[junit4] 2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
[junit4] 2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162522 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
[junit4] 2> 162522 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=null: minGen=9223372036854775807 packetCount=0
[junit4] 2> 162522 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
[junit4] 2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: startCommit(): start
[junit4] 2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: startCommit index= changeCount=1
[junit4] 2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: done all syncs: []
[junit4] 2> 162523 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: pendingCommit != null
[junit4] 2> 162524 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: wrote segments file "segments_1"
[junit4] 2> 162524 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = true]
[junit4] 2> 162524 T6296 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
[junit4] 2> 162524 T6296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 162525 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
[junit4] 2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: took 3.6 msec
[junit4] 2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: commit: done
[junit4] 2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback
[junit4] 2> 162525 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: all running merges have aborted
[junit4] 2> 162526 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback: done finish merges
[junit4] 2> 162526 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: abort
[junit4] 2> 162526 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: done abort; abortedFiles=[] success=true
[junit4] 2> 162526 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: rollback: infos=
[junit4] 2> 162526 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
[junit4] 2> 162527 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
[junit4] 2> 162527 T6296 oasc.SolrCore.initWriters created json: solr.JSONResponseWriter
[junit4] 2> 162527 T6296 oasc.SolrCore.initWriters adding lazy queryResponseWriter: solr.VelocityResponseWriter
[junit4] 2> 162527 T6296 oasc.SolrCore.initWriters created velocity: solr.VelocityResponseWriter
[junit4] 2> 162528 T6296 oasc.SolrCore.initWriters created xslt: solr.XSLTResponseWriter
[junit4] 2> 162528 T6296 oasr.XSLTResponseWriter.init xsltCacheLifetimeSeconds=5
[junit4] 2> 162529 T6296 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 162529 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 162530 T6296 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
[junit4] 2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /query: solr.SearchHandler
[junit4] 2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig created /browse: solr.SearchHandler
[junit4] 2> 162531 T6296 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.extraction.ExtractingRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /update/extract: solr.extraction.ExtractingRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
[junit4] 2> 162532 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.DocumentAnalysisRequestHandler
[junit4] 2> 162533 T6296 oasc.RequestHandlers.initHandlersFromConfig created /analysis/document: solr.DocumentAnalysisRequestHandler
[junit4] 2> 162533 T6296 oasc.RequestHandlers.initHandlersFromConfig created /admin/: solr.admin.AdminHandlers
[junit4] 2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
[junit4] 2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 162534 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /spell: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /suggest: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig created /tvrh: solr.SearchHandler
[junit4] 2> 162535 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
[junit4] 2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig created /terms: solr.SearchHandler
[junit4] 2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.SearchHandler
[junit4] 2> 162536 T6296 oasc.RequestHandlers.initHandlersFromConfig created /elevate: solr.SearchHandler
[junit4] 2> 162547 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 162548 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 162550 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 162551 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 162552 T6296 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 162556 T6296 oasu.CommitTracker.<init> Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 162556 T6296 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 162557 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: current segments file is "segments_1"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@44dd1ac2
[junit4] 2> 162558 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: init: load commit "segments_1"
[junit4] 2> 162558 T6296 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
[junit4] 2> 162558 T6296 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 162558 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
[junit4] 2> 162559 T6296 oasu.LoggingInfoStream.message [IFD][coreLoadExecutor-200-thread-1]: 0 msec to checkpoint
[junit4] 2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: init: create=false
[junit4] 2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]:
[junit4] 2> dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index)
[junit4] 2> index=
[junit4] 2> version=5.0-SNAPSHOT
[junit4] 2> matchVersion=LUCENE_5_0
[junit4] 2> analyzer=null
[junit4] 2> ramBufferSizeMB=100.0
[junit4] 2> maxBufferedDocs=-1
[junit4] 2> maxBufferedDeleteTerms=-1
[junit4] 2> mergedSegmentWarmer=null
[junit4] 2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
[junit4] 2> commit=null
[junit4] 2> openMode=APPEND
[junit4] 2> similarity=org.apache.lucene.search.similarities.DefaultSimilarity
[junit4] 2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=1, maxMergeCount=2, mergeThreadPriority=-1
[junit4] 2> default WRITE_LOCK_TIMEOUT=1000
[junit4] 2> writeLockTimeout=1000
[junit4] 2> codec=Lucene49
[junit4] 2> infoStream=org.apache.solr.update.LoggingInfoStream
[junit4] 2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@489c9308
[junit4] 2> readerPooling=false
[junit4] 2> perThreadHardLimitMB=1945
[junit4] 2> useCompoundFile=false
[junit4] 2> checkIntegrityAtMerge=false
[junit4] 2> writer=org.apache.lucene.util.SetOnce@1f791776
[junit4] 2>
[junit4] 2> 162559 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: flush at getReader
[junit4] 2> 162560 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: startFullFlush
[junit4] 2> 162560 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162560 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: apply all deletes during flush
[junit4] 2> 162560 T6296 oasu.LoggingInfoStream.message [BD][coreLoadExecutor-200-thread-1]: prune sis=segments_1: minGen=9223372036854775807 packetCount=0
[junit4] 2> 162560 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: return reader version=1 reader=StandardDirectoryReader(segments_1:1:nrt)
[junit4] 2> 162561 T6296 oasu.LoggingInfoStream.message [DW][coreLoadExecutor-200-thread-1]: coreLoadExecutor-200-thread-1 finishFullFlush success=true
[junit4] 2> 162561 T6296 oasu.LoggingInfoStream.message [IW][coreLoadExecutor-200-thread-1]: getReader took 2 msec
[junit4] 2> 162561 T6296 oass.SolrIndexSearcher.<init> Opening Searcher@1117a1f6[collection1] main
[junit4] 2> 162562 T6296 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/conf; switching to use InMemory storage instead.
[junit4] 2> 162562 T6296 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 162562 T6296 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 162562 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 162563 T6296 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
[junit4] 2> 162563 T6296 oasr.RestManager.init Initializing 2 registered ManagedResources
[junit4] 2> 162563 T6296 oasr.ManagedResourceStorage.load Reading _schema_analysis_stopwords_english.json using InMemoryStorage
[junit4] 2> 162563 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /schema/analysis/stopwords/english
[junit4] 2> 162563 T6296 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _schema_analysis_stopwords_english.json using InMemoryStorage
[junit4] 2> 162564 T6296 oasrsa.ManagedWordSetResource.onManagedDataLoadedFromStorage Loaded 0 words for /schema/analysis/stopwords/english
[junit4] 2> 162564 T6296 oasr.ManagedResource.notifyObserversDuringInit Notified 1 observers of /schema/analysis/stopwords/english
[junit4] 2> 162564 T6296 oasr.ManagedResourceStorage.load Reading _schema_analysis_synonyms_english.json using InMemoryStorage
[junit4] 2> 162564 T6296 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /schema/analysis/synonyms/english
[junit4] 2> 162564 T6296 oasrsa.ManagedSynonymFilterFactory$SynonymManager.onManagedDataLoadedFromStorage Loaded 0 synonym mappings for /schema/analysis/synonyms/english
[junit4] 2> 162565 T6296 oasr.ManagedResource.notifyObserversDuringInit Notified 1 observers of /schema/analysis/synonyms/english
[junit4] 2> 162565 T6296 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 162565 T6296 oass.DirectSolrSpellChecker.init init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
[junit4] 2> 162565 T6296 oashc.SpellCheckComponent.inform No queryConverter defined, using default converter
[junit4] 2> 162566 T6296 oashc.SuggestComponent.inform Initializing SuggestComponent
[junit4] 2> 162566 T6296 oasss.SolrSuggester.init init: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
[junit4] 2> 162566 T6296 oasss.SolrSuggester.init Dictionary loaded with params: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string}
[junit4] 2> 162567 T6296 oashc.QueryElevationComponent.inform Loading QueryElevation from: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/example/solr/collection1/conf/elevate.xml
[junit4] 2> 162577 T6296 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 162577 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@1117a1f6[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 162577 T6296 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 162578 T6286 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1
[junit4] 2> 162578 T6286 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> ASYNC NEW_CORE C209 name=collection1 org.apache.solr.core.SolrCore@4fda2aac
[junit4] 2> 162579 T6297 C209 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher} hits=0 status=0 QTime=2
[junit4] 2> 162579 T6286 oas.SolrJettyTestBase.createJetty Jetty Assigned Port#59691
[junit4] 2> 162579 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
[junit4] 2> 162580 T6297 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
[junit4] 2> 162580 T6297 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
[junit4] 2> 162580 T6297 oashc.SuggestComponent$SuggesterListener.newSearcher Loading suggester index for: mySuggester
[junit4] 2> 162581 T6297 oasss.SolrSuggester.reload reload()
[junit4] 2> 162581 T6297 oasss.SolrSuggester.build build()
[junit4] 2> 162585 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1117a1f6[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 162590 T6286 oas.SolrTestCaseJ4.setUp ###Starting testChineseDefaults
[junit4] 2> 162595 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162596 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: true; BD changes: false
[junit4] 2> 162596 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162597 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
[junit4] 2> 162597 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 162598 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162598 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289: flush naked frozen global deletes
[junit4] 2> 162599 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes 1 deleted queries bytesUsed=64 delGen=2 packetCount=1 totBytesUsed=64
[junit4] 2> 162599 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: flush: push buffered updates: 1 deleted queries bytesUsed=64
[junit4] 2> 162600 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 162600 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_1: minGen=9223372036854775807 packetCount=1
[junit4] 2> 162600 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
[junit4] 2> 162601 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=1 reader=StandardDirectoryReader(segments_1:1:nrt)
[junit4] 2> 162601 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 162602 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 5 msec
[junit4] 2> 162603 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {deleteByQuery=*:* (-1475865354122035200)} 0 10
[junit4] 2> 162606 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 162607 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162607 T6289 C209 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 162608 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162608 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
[junit4] 2> 162608 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162609 T6289 C209 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
[junit4] 2> 162609 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 162610 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 4
[junit4] 2> 162613 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&version=2.2&wt=xml} hits=0 status=0 QTime=0
[junit4] 2> 162614 T6286 oascsi.ConcurrentUpdateSolrServer.request blocking for commit/optimize
[junit4] 2> 162620 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 162620 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
[junit4] 2> 162621 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
[junit4] 2> 162621 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
[junit4] 2> 162622 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
[junit4] 2> 162622 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: index before flush
[junit4] 2> 162622 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 162623 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=1 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162623 T6289 C209 oasu.LoggingInfoStream.message [DWFC][qtp1189521383-6289]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0, segment=_0, aborting=false, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 2 ]]
[junit4] 2> 162624 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flush postings as segment _0 numDocs=1
[junit4] 2> 162627 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has 0 deleted docs
[junit4] 2> 162628 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has no vectors; norms; no docValues; prox; freqs
[junit4] 2> 162628 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushedFiles=[_0_Lucene41_0.tip, _0_Lucene41_0.tim, _0.fnm, _0.nvm, _0.fdx, _0.nvd, _0.fdt, _0_Lucene41_0.pos, _0_Lucene41_0.doc]
[junit4] 2> 162629 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed codec=Lucene49
[junit4] 2> 162629 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed: segment=_0 ramUsed=0.071 MB newFlushedSize(includes docstores)=0.001 MB docs/MB=979.063
[junit4] 2> 162630 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: publishFlushedSegment seg-private updates=null
[junit4] 2> 162631 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publishFlushedSegment
[junit4] 2> 162631 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes 1 deleted terms (unique count=1) bytesUsed=1056 delGen=4 packetCount=1 totBytesUsed=1056
[junit4] 2> 162631 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publish sets newSegment delGen=5 seg=_0(5.0):C1
[junit4] 2> 162632 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1" [1 segments ; isCommit = false]
[junit4] 2> 162632 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
[junit4] 2> 162633 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 162633 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_0(5.0):C1] packetCount=1
[junit4] 2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 1 msec
[junit4] 2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_1: _0(5.0):C1 minGen=6 packetCount=1
[junit4] 2> 162634 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
[junit4] 2> 162635 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 162635 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: findMerges: 1 segments
[junit4] 2> 162636 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: seg=_0(5.0):C1 size=0.001 MB [floored]
[junit4] 2> 162636 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: allowedSegmentCount=1 vs count=1 (eligible count=1) tooBigCount=0
[junit4] 2> 162637 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: now merge
[junit4] 2> 162637 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: index: _0(5.0):C1
[junit4] 2> 162638 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: no more merges pending; now return
[junit4] 2> 162638 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
[junit4] 2> 162638 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index=_0(5.0):C1 changeCount=3
[junit4] 2> 162639 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: [_0_Lucene41_0.tip, _0.si, _0_Lucene41_0.tim, _0.nvm, _0.fnm, _0.fdx, _0.nvd, _0.fdt, _0_Lucene41_0.doc, _0_Lucene41_0.pos]
[junit4] 2> 162640 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
[junit4] 2> 162640 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_2"
[junit4] 2> 162641 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1" [1 segments ; isCommit = true]
[junit4] 2> 162641 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_2,generation=2}
[junit4] 2> 162642 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 162642 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_1"
[junit4] 2> 162643 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_1"
[junit4] 2> 162643 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 2 msec to checkpoint
[junit4] 2> 162644 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 22.2 msec
[junit4] 2> 162644 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
[junit4] 2> 162645 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162645 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: false; DW changes: false; BD changes: false
[junit4] 2> 162646 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
[junit4] 2> 162646 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 162646 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162647 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 162647 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
[junit4] 2> 162648 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2: _0(5.0):C1 minGen=6 packetCount=0
[junit4] 2> 162650 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=3 reader=StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)
[junit4] 2> 162650 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 162650 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 4 msec
[junit4] 2> 162651 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@755b797[collection1] main
[junit4] 2> 162652 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 162652 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@755b797[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
[junit4] 2> 162653 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
[junit4] 2> 162653 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@755b797[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
[junit4] 2> 162653 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {add=[42 (1475865354146152448)],commit=} 0 36
[junit4] 2> 162659 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=????&version=2.2&wt=xml} hits=1 status=0 QTime=2
[junit4] 2> 162661 T6286 oas.SolrTestCaseJ4.tearDown ###Ending testChineseDefaults
[junit4] 2> 162674 T6286 oas.SolrTestCaseJ4.setUp ###Starting testUpdateField
[junit4] 2> 162679 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162679 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: true; BD changes: false
[junit4] 2> 162680 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162680 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
[junit4] 2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162681 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289: flush naked frozen global deletes
[junit4] 2> 162682 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes 1 deleted queries bytesUsed=64 delGen=8 packetCount=1 totBytesUsed=64
[junit4] 2> 162682 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: flush: push buffered updates: 1 deleted queries bytesUsed=64
[junit4] 2> 162683 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 162683 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_0(5.0):C1] packetCount=1
[junit4] 2> 162854 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: seg=_0(5.0):C1 segGen=6 coalesced deletes=[CoalescedUpdates(termSets=1,queries=1,numericDVUpdates=0,binaryDVUpdates=0)] newDelCount=1 100% deleted
[junit4] 2> 162855 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 172 msec
[junit4] 2> 162856 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_0(5.0):C1/1" [1 segments ; isCommit = false]
[junit4] 2> 162856 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
[junit4] 2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: drop 100% deleted segments: _0(5.0):C1/1
[junit4] 2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "" [0 segments ; isCommit = false]
[junit4] 2> 162857 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
[junit4] 2> 162858 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2: minGen=9223372036854775807 packetCount=1
[junit4] 2> 162858 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
[junit4] 2> 162859 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=5 reader=StandardDirectoryReader(segments_2:5:nrt)
[junit4] 2> 162859 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 162860 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 180 msec
[junit4] 2> 162860 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@1119083d[collection1] realtime
[junit4] 2> 162861 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {deleteByQuery=*:* (-1475865354209067008)} 0 184
[junit4] 2> 162865 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
[junit4] 2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
[junit4] 2> 162866 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
[junit4] 2> 162867 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
[junit4] 2> 162867 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: index before flush
[junit4] 2> 162868 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 162868 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162868 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 162869 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_2: minGen=9223372036854775807 packetCount=0
[junit4] 2> 162869 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 162870 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
[junit4] 2> 162870 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index= changeCount=6
[junit4] 2> 162871 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: []
[junit4] 2> 162871 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
[junit4] 2> 162872 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_3"
[junit4] 2> 162872 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "" [0 segments ; isCommit = true]
[junit4] 2> 162873 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_2,generation=2}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_3,generation=3}
[junit4] 2> 162873 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4] 2> 162874 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_2"
[junit4] 2> 162874 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_2"
[junit4] 2> 162875 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 2 msec to checkpoint
[junit4] 2> 162875 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 8.1 msec
[junit4] 2> 162876 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
[junit4] 2> 162876 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162876 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
[junit4] 2> 162877 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 162877 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@2776bcc3[collection1] main
[junit4] 2> 162878 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 162878 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@2776bcc3[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 162879 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
[junit4] 2> 162879 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2776bcc3[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.tip"
[junit4] 2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.si"
[junit4] 2> 162880 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.tim"
[junit4] 2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.nvm"
[junit4] 2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fnm"
[junit4] 2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fdx"
[junit4] 2> 162881 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.nvd"
[junit4] 2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0.fdt"
[junit4] 2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.doc"
[junit4] 2> 162882 T6297 oasu.LoggingInfoStream.message [IFD][searcherExecutor-201-thread-1]: delete "_0_Lucene41_0.pos"
[junit4] 2> 162882 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 17
[junit4] 2> 163137 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {add=[unique (1475865354429267968)]} 0 250
[junit4] 2> 163140 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 163140 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
[junit4] 2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: enter lock
[junit4] 2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: now prepare
[junit4] 2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: prepareCommit: flush
[junit4] 2> 163141 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: index before flush
[junit4] 2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=1 deletes=true hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163142 T6289 C209 oasu.LoggingInfoStream.message [DWFC][qtp1189521383-6289]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0, segment=_1, aborting=false, numDocsInRAM=1, deleteQueue=DWDQ: [ generation: 6 ]]
[junit4] 2> 163143 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flush postings as segment _1 numDocs=1
[junit4] 2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has 0 deleted docs
[junit4] 2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: new segment has no vectors; norms; no docValues; prox; freqs
[junit4] 2> 163145 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushedFiles=[_1.nvd, _1_Lucene41_0.doc, _1_Lucene41_0.tip, _1_Lucene41_0.tim, _1.nvm, _1.fdt, _1.fdx, _1.fnm, _1_Lucene41_0.pos]
[junit4] 2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed codec=Lucene49
[junit4] 2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DWPT][qtp1189521383-6289]: flushed: segment=_1 ramUsed=0.079 MB newFlushedSize(includes docstores)=0.001 MB docs/MB=757.642
[junit4] 2> 163146 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: publishFlushedSegment seg-private updates=null
[junit4] 2> 163147 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publishFlushedSegment
[junit4] 2> 163147 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: push deletes 1 deleted terms (unique count=1) bytesUsed=1056 delGen=11 packetCount=1 totBytesUsed=1056
[junit4] 2> 163147 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: publish sets newSegment delGen=12 seg=_1(5.0):C1
[junit4] 2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_1(5.0):C1" [1 segments ; isCommit = false]
[junit4] 2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 0 msec to checkpoint
[junit4] 2> 163148 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: infos=[_1(5.0):C1] packetCount=1
[junit4] 2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes took 0 msec
[junit4] 2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_3: _1(5.0):C1 minGen=13 packetCount=1
[junit4] 2> 163149 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: pruneDeletes: prune 1 packets; 0 packets remain
[junit4] 2> 163150 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 163150 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: findMerges: 1 segments
[junit4] 2> 163150 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: seg=_1(5.0):C1 size=0.001 MB [floored]
[junit4] 2> 163151 T6289 C209 oasu.LoggingInfoStream.message [TMP][qtp1189521383-6289]: allowedSegmentCount=1 vs count=1 (eligible count=1) tooBigCount=0
[junit4] 2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: now merge
[junit4] 2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: index: _1(5.0):C1
[junit4] 2> 163151 T6289 C209 oasu.LoggingInfoStream.message [CMS][qtp1189521383-6289]: no more merges pending; now return
[junit4] 2> 163152 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit(): start
[junit4] 2> 163152 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: startCommit index=_1(5.0):C1 changeCount=9
[junit4] 2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: done all syncs: [_1.nvd, _1_Lucene41_0.doc, _1_Lucene41_0.tip, _1.fdt, _1.nvm, _1_Lucene41_0.tim, _1.si, _1.fdx, _1_Lucene41_0.pos, _1.fnm]
[junit4] 2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: pendingCommit != null
[junit4] 2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: wrote segments file "segments_4"
[junit4] 2> 163153 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: now checkpoint "_1(5.0):C1" [1 segments ; isCommit = true]
[junit4] 2> 163154 T6289 C209 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_3,generation=3}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1c06dda5 lockFactory=NativeFSLockFactory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index),segFN=segments_4,generation=4}
[junit4] 2> 163154 T6289 C209 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4] 2> 163154 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: deleteCommits: now decRef commit "segments_3"
[junit4] 2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: delete "segments_3"
[junit4] 2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IFD][qtp1189521383-6289]: 1 msec to checkpoint
[junit4] 2> 163155 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: took 14.2 msec
[junit4] 2> 163156 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: done
[junit4] 2> 163156 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163156 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: false; DW changes: false; BD changes: false
[junit4] 2> 163157 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: flush at getReader
[junit4] 2> 163157 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: startFullFlush
[junit4] 2> 163157 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163157 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 163158 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
[junit4] 2> 163158 T6289 C209 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_4: _1(5.0):C1 minGen=13 packetCount=0
[junit4] 2> 163159 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=7 reader=StandardDirectoryReader(segments_4:7:nrt _1(5.0):C1)
[junit4] 2> 163160 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 163160 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 3 msec
[junit4] 2> 163160 T6289 C209 oass.SolrIndexSearcher.<init> Opening Searcher@76e4acbe[collection1] main
[junit4] 2> 163161 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 163161 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@76e4acbe[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0):C1))}
[junit4] 2> 163161 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
[junit4] 2> 163162 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@76e4acbe[collection1] main{UninvertingDirectoryReader(Uninverting(_1(5.0):C1))}
[junit4] 2> 163162 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 22
[junit4] 2> 163173 T6289 C209 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&fl=id,price_f,name,_version_&version=2.2&wt=xml} hits=1 status=0 QTime=8
[junit4] 2> 163179 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {} 0 2
[junit4] 2> 163430 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 163431 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163431 T6289 C209 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 163431 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163432 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: nrtIsCurrent: infoVersion matches: true; DW changes: false; BD changes: false
[junit4] 2> 163432 T6289 C209 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 163432 T6289 C209 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
[junit4] 2> 163433 T6289 C209 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 163433 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 3
[junit4] 2> 163688 T6289 C209 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2.2&wt=xml} {add=[unique (1475865355007033344)]} 0 251
[junit4] 2> 163692 T6289 C209 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 163693 T6289 C209 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: commit: start
[jun
[...truncated too long message...]
2> 183797 T6289 C211 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: anyChanges? numDocsInRam=0 deletes=false hasTickets:false pendingChangesInFullFlush: false
[junit4] 2> 183797 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: apply all deletes during flush
[junit4] 2> 183798 T6289 C211 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: applyDeletes: no deletes; skipping
[junit4] 2> 183798 T6289 C211 oasu.LoggingInfoStream.message [BD][qtp1189521383-6289]: prune sis=segments_20: _25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2 minGen=338 packetCount=0
[junit4] 2> 183806 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: return reader version=239 reader=StandardDirectoryReader(segments_20:239:nrt _25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2)
[junit4] 2> 183806 T6289 C211 oasu.LoggingInfoStream.message [DW][qtp1189521383-6289]: qtp1189521383-6289 finishFullFlush success=true
[junit4] 2> 183807 T6289 C211 oasu.LoggingInfoStream.message [IW][qtp1189521383-6289]: getReader took 11 msec
[junit4] 2> 183808 T6289 C211 oass.SolrIndexSearcher.<init> Opening Searcher@132c9694[collection1] main
[junit4] 2> 183808 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener sending requests to Searcher@132c9694[collection1] main{UninvertingDirectoryReader(Uninverting(_25(5.0):C2) Uninverting(_22(5.0):C2) Uninverting(_26(5.0):C1) Uninverting(_23(5.0):C3) Uninverting(_24(5.0):C2))}
[junit4] 2> 183809 T6297 oasc.QuerySenderListener.newSearcher QuerySenderListener done.
[junit4] 2> 183809 T6297 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@132c9694[collection1] main{UninvertingDirectoryReader(Uninverting(_25(5.0):C2) Uninverting(_22(5.0):C2) Uninverting(_26(5.0):C1) Uninverting(_23(5.0):C3) Uninverting(_24(5.0):C2))}
[junit4] 2> 183809 T6289 C211 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 183810 T6289 C211 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&softCommit=false&version=2.2&wt=xml&commit=true} {commit=} 0 68
[junit4] 2> 183814 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=inStock:true&version=2.2&wt=xml} hits=5 status=0 QTime=1
[junit4] 2> 183819 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=inStock:false&version=2.2&wt=xml} hits=5 status=0 QTime=1
[junit4] 2> 183861 T6289 C211 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={stats.field=val_i&rows=0&q=*:*&version=2.2&wt=xml&stats=true&f.val_i.stats.facet=inStock} hits=10 status=0 QTime=38
[junit4] 2> 183863 T6286 oas.SolrTestCaseJ4.tearDown ###Ending testStatistics
[junit4] 2> 183866 T6286 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=761401314
[junit4] 2> 183867 T6286 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4fda2aac
[junit4] 2> 183871 T6286 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=75,autocommit maxTime=15000ms,autocommits=0,soft autocommits=3,optimizes=8,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=359,cumulative_deletesById=5,cumulative_deletesByQuery=34,cumulative_errors=1,transaction_logs_total_size=3981,transaction_logs_total_number=10}
[junit4] 2> 183871 T6286 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 183872 T6286 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 183872 T6286 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 183873 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: waitForMerges
[junit4] 2> 183873 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: waitForMerges done
[junit4] 2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback
[junit4] 2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: all running merges have aborted
[junit4] 2> 183874 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback: done finish merges
[junit4] 2> 183875 T6286 C211 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: abort
[junit4] 2> 183875 T6286 C211 oasu.LoggingInfoStream.message [DW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: done abort; abortedFiles=[] success=true
[junit4] 2> 183876 T6286 C211 oasu.LoggingInfoStream.message [IW][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: rollback: infos=_25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2
[junit4] 2> 183876 T6286 C211 oasu.LoggingInfoStream.message [IFD][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: now checkpoint "_25(5.0):C2 _22(5.0):C2 _26(5.0):C1 _23(5.0):C3 _24(5.0):C2" [5 segments ; isCommit = false]
[junit4] 2> 183877 T6286 C211 oasu.LoggingInfoStream.message [IFD][SUITE-SolrExampleStreamingTest-seed#[25EEF9A8D20C97FB]-worker]: 0 msec to checkpoint
[junit4] 2> 183878 T6286 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 183879 T6286 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 183880 T6286 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001;done=false>>]
[junit4] 2> 183880 T6286 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001
[junit4] 2> 183880 T6286 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index;done=false>>]
[junit4] 2> 183881 T6286 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001/tempDir-001/index
[junit4] 2> 183881 T6286 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
[junit4] 2> 188870 T6286 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.SolrExampleStreamingTest-25EEF9A8D20C97FB-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=tr_TR, timezone=America/Buenos_Aires
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=83553456,total=268435456
[junit4] 2> NOTE: All tests run in this JVM: [ExternalHttpClientTest, MergeIndexesEmbeddedTest, TestSolrProperties, TestLBHttpSolrServer, BasicHttpSolrServerTest, FieldAnalysisResponseTest, QueryResponseTest, ConcurrentUpdateSolrServerTest, NoOpResponseParserTest, SolrExampleStreamingTest]
[junit4] Completed on J1 in 27.50s, 28 tests, 1 error <<< FAILURES!
[...truncated 79 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:202: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 51 suites, 299 tests, 1 error
Total time: 112 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 10361750 bytes
Compression is 0.0%
Took 3.1 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4791 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4791/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([9BBB3C28E81C7420:1A5DB2309F43141C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 13795 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/init-core-data-001
[junit4] 2> 3161990 T8786 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 3161990 T8786 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_x/c
[junit4] 2> 3161996 T8786 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 3161997 T8786 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3161997 T8787 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 3162097 T8786 oasc.ZkTestServer.run start zk server on port:37683
[junit4] 2> 3162098 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3162103 T8793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24f55b5f name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3162103 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3162104 T8786 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 3162107 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3162108 T8795 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f20083b name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3162108 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3162109 T8786 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 3162111 T8786 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 3162113 T8786 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 3162114 T8786 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 3162116 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3162117 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 3162119 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 3162120 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 3162223 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3162223 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3162226 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3162226 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 3162228 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3162229 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 3162231 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3162232 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 3162234 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3162234 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 3162236 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3162237 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 3162239 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3162239 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3162241 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3162242 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 3162244 T8786 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3162245 T8786 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 3163457 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3163462 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65517
[junit4] 2> 3163463 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3163463 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3163463 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002
[junit4] 2> 3163464 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/'
[junit4] 2> 3163518 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/solr.xml
[junit4] 2> 3163600 T8786 oasc.CoreContainer.<init> New CoreContainer 1435050643
[junit4] 2> 3163601 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/]
[junit4] 2> 3163602 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3163603 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3163603 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3163604 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3163605 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3163605 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3163606 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3163606 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3163607 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3163607 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3163608 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3163608 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
[junit4] 2> 3163608 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3163609 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3163611 T8806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e3d0cc4 name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3163611 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3163613 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3163614 T8808 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c44a9cd name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3163615 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3163616 T8786 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 3163618 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 3163620 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 3163622 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 3163624 T8786 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 3163626 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 3163627 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65517__x%2Fc
[junit4] 2> 3163628 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65517__x%2Fc
[junit4] 2> 3163630 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 3163631 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 3163634 T8786 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:65517__x%2Fc
[junit4] 2> 3163634 T8786 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 3163635 T8786 oasc.Overseer.start Overseer (id=92236290143027203-127.0.0.1:65517__x%2Fc-n_0000000000) starting
[junit4] 2> 3163637 T8786 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 3163643 T8810 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 3163643 T8786 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 3163645 T8786 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 3163646 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3163648 T8809 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 3163652 T8811 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 3163652 T8811 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3163653 T8811 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3163653 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3163654 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:65517/_x/c",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:65517__x%2Fc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 3163655 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3163657 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3163657 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3164653 T8811 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3164654 T8811 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 3164655 T8811 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3164655 T8811 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3164656 T8811 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3164656 T8811 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/'
[junit4] 2> 3164658 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 3164658 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 3164659 T8811 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 3164733 T8811 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3164784 T8811 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3164885 T8811 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3164898 T8811 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3165306 T8811 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3165319 T8811 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3165322 T8811 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3165334 T8811 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3165338 T8811 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3165342 T8811 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3165343 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3165343 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3165344 T8811 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3165345 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3165345 T8811 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3165345 T8811 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 3165345 T8811 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3165346 T8811 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/
[junit4] 2> 3165346 T8811 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
[junit4] 2> 3165347 T8811 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data
[junit4] 2> 3165347 T8811 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index/
[junit4] 2> 3165347 T8811 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 3165348 T8811 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-001/control/data/index
[junit4] 2> 3165348 T8811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
[junit4] 2> 3165349 T8811 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@42cdcd4b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@79789efc),segFN=segments_1,generation=1}
[junit4] 2> 3165350 T8811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3165354 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3165355 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3165355 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3165356 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3165357 T8811 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3165357 T8811 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3165358 T8811 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3165358 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3165359 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3165359 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3165360 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3165360 T8811 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3165361 T8811 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3165362 T8811 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3165362 T8811 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3165363 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3165363 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3165364 T8811 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3165382 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3165384 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3165386 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3165389 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3165391 T8811 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3165397 T8811 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3165397 T8811 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3165398 T8811 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3165399 T8811 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@42cdcd4b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@79789efc),segFN=segments_1,generation=1}
[junit4] 2> 3165399 T8811 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3165399 T8811 oass.SolrIndexSearcher.<init> Opening Searcher@2d88cd47[collection1] main
[junit4] 2> 3165400 T8811 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3165401 T8811 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3165401 T8811 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3165401 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3165402 T8811 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3165402 T8811 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3165402 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 3165403 T8811 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3165403 T8811 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 3165405 T8811 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 3165405 T8811 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3165406 T8811 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3165408 T8813 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2d88cd47[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3165412 T8811 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3165412 T8816 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65517/_x/c collection:control_collection shard:shard1
[junit4] 2> 3165413 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 3165413 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3165413 T8816 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 3165415 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3165416 T8818 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23e0a6d1 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3165416 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3165419 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3165420 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3165421 T8786 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3165422 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165422 T8816 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3165422 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1447 name=collection1 org.apache.solr.core.SolrCore@1c4d38c1 url=http://127.0.0.1:65517/_x/c/collection1 node=127.0.0.1:65517__x%2Fc C1447_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:65517/_x/c, core=collection1, node_name=127.0.0.1:65517__x%2Fc}
[junit4] 2> 3165422 T8816 C1447 P65517 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65517/_x/c/collection1/
[junit4] 2> 3165423 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3165423 T8816 C1447 P65517 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3165423 T8816 C1447 P65517 oasc.SyncStrategy.syncToMe http://127.0.0.1:65517/_x/c/collection1/ has no replicas
[junit4] 2> 3165423 T8816 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65517/_x/c/collection1/ shard1
[junit4] 2> 3165424 T8816 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 3165425 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165425 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3165425 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3165428 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3165430 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165531 T8812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3165531 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3165580 T8816 oasc.ZkController.register We are http://127.0.0.1:65517/_x/c/collection1/ and leader is http://127.0.0.1:65517/_x/c/collection1/
[junit4] 2> 3165581 T8816 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65517/_x/c
[junit4] 2> 3165581 T8816 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3165581 T8816 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 3165581 T8816 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3165582 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165583 T8816 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3165583 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165583 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165584 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3165585 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:65517/_x/c",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:65517__x%2Fc",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3165586 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3165688 T8820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3165688 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3166592 T8786 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 3166593 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3166595 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:63043
[junit4] 2> 3166597 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3166597 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3166597 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003
[junit4] 2> 3166598 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/'
[junit4] 2> 3166635 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/solr.xml
[junit4] 2> 3166690 T8786 oasc.CoreContainer.<init> New CoreContainer 1726624691
[junit4] 2> 3166690 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/]
[junit4] 2> 3166691 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3166692 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3166693 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3166694 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3166694 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3166695 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3166695 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3166695 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
[junit4] 2> 3166695 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3166696 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3166698 T8832 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19e8618e name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3166698 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3166701 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3166702 T8834 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b42a3c7 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3166702 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3166707 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3167710 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63043__x%2Fc
[junit4] 2> 3167711 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63043__x%2Fc
[junit4] 2> 3167722 T8836 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3167722 T8836 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3167723 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3167723 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3167723 T8836 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3167723 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3167724 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63043/_x/c",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:63043__x%2Fc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 3167725 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3167727 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3167727 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3167727 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3167727 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3168724 T8836 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3168725 T8836 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3168726 T8836 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3168726 T8836 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3168727 T8836 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3168727 T8836 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/'
[junit4] 2> 3168728 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 3168729 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 3168730 T8836 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 3168850 T8836 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3168901 T8836 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3169003 T8836 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3169016 T8836 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3169461 T8836 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3169478 T8836 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3169482 T8836 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3169493 T8836 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3169498 T8836 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3169503 T8836 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3169504 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3169506 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3169506 T8836 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3169507 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3169507 T8836 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3169507 T8836 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3169508 T8836 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3169508 T8836 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/
[junit4] 2> 3169508 T8836 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
[junit4] 2> 3169509 T8836 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data
[junit4] 2> 3169510 T8836 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index/
[junit4] 2> 3169510 T8836 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3169510 T8836 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-003/collection1/data/index
[junit4] 2> 3169511 T8836 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
[junit4] 2> 3169512 T8836 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@14c2ae6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63aadd07),segFN=segments_1,generation=1}
[junit4] 2> 3169512 T8836 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3169517 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3169518 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3169519 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3169520 T8836 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3169521 T8836 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3169521 T8836 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3169522 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3169522 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3169523 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3169523 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3169524 T8836 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3169525 T8836 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3169526 T8836 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3169526 T8836 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3169527 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3169527 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3169528 T8836 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3169555 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3169557 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3169559 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3169562 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3169564 T8836 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3169571 T8836 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3169571 T8836 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3169572 T8836 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3169573 T8836 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@14c2ae6a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63aadd07),segFN=segments_1,generation=1}
[junit4] 2> 3169573 T8836 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3169573 T8836 oass.SolrIndexSearcher.<init> Opening Searcher@58c32608[collection1] main
[junit4] 2> 3169574 T8836 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3169575 T8836 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3169575 T8836 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3169575 T8836 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3169576 T8836 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3169576 T8836 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3169577 T8836 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3169577 T8836 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3169577 T8836 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3169578 T8836 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3169582 T8837 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58c32608[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3169585 T8836 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3169586 T8840 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:63043/_x/c collection:collection1 shard:shard1
[junit4] 2> 3169586 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 3169587 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3169587 T8840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 3169591 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3169592 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169592 T8840 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3169592 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1448 name=collection1 org.apache.solr.core.SolrCore@27a84924 url=http://127.0.0.1:63043/_x/c/collection1 node=127.0.0.1:63043__x%2Fc C1448_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:63043/_x/c, core=collection1, node_name=127.0.0.1:63043__x%2Fc}
[junit4] 2> 3169592 T8840 C1448 P63043 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:63043/_x/c/collection1/
[junit4] 2> 3169593 T8840 C1448 P63043 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3169593 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3169593 T8840 C1448 P63043 oasc.SyncStrategy.syncToMe http://127.0.0.1:63043/_x/c/collection1/ has no replicas
[junit4] 2> 3169593 T8840 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:63043/_x/c/collection1/ shard1
[junit4] 2> 3169593 T8840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 3169595 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169595 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169595 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169595 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169597 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3169599 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169701 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169701 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169701 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169750 T8840 oasc.ZkController.register We are http://127.0.0.1:63043/_x/c/collection1/ and leader is http://127.0.0.1:63043/_x/c/collection1/
[junit4] 2> 3169751 T8840 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:63043/_x/c
[junit4] 2> 3169751 T8840 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3169751 T8840 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3169751 T8840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3169752 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169752 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169752 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169753 T8840 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3169753 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3169754 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:63043/_x/c",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:63043__x%2Fc",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3169755 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3169858 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169858 T8820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3169858 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3171349 T8786 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 3171349 T8786 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3171353 T8786 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10100
[junit4] 2> 3171353 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3171353 T8786 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3171354 T8786 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004
[junit4] 2> 3171354 T8786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/'
[junit4] 2> 3171405 T8786 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/solr.xml
[junit4] 2> 3171481 T8786 oasc.CoreContainer.<init> New CoreContainer 1256265506
[junit4] 2> 3171482 T8786 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/]
[junit4] 2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3171483 T8786 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3171484 T8786 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3171485 T8786 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3171485 T8786 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3171485 T8786 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3171486 T8786 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3171486 T8786 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3171486 T8786 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3171486 T8786 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37683/solr
[junit4] 2> 3171487 T8786 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3171488 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3171489 T8851 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fb5df95 name:ZooKeeperConnection Watcher:127.0.0.1:37683 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3171490 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3171492 T8786 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3171493 T8853 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d856391 name:ZooKeeperConnection Watcher:127.0.0.1:37683/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3171493 T8786 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3171497 T8786 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3172500 T8786 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10100__x%2Fc
[junit4] 2> 3172501 T8786 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10100__x%2Fc
[junit4] 2> 3172512 T8855 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3172513 T8855 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3172513 T8820 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3172513 T8812 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3172513 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3172513 T8855 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3172515 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:10100/_x/c",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10100__x%2Fc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 3172516 T8809 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 3172518 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3172518 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3172518 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3172518 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3172518 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3173515 T8855 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3173516 T8855 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3173517 T8855 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3173517 T8855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3173517 T8855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3173518 T8855 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/'
[junit4] 2> 3173519 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 3173520 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 3173520 T8855 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 3173575 T8855 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3173612 T8855 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3173752 T8855 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3173760 T8855 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3174044 T8855 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3174054 T8855 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3174056 T8855 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3174065 T8855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3174069 T8855 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3174072 T8855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3174073 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3174074 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3174075 T8855 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3174076 T8855 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3174076 T8855 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3174076 T8855 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/
[junit4] 2> 3174076 T8855 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b4fab9a
[junit4] 2> 3174077 T8855 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data
[junit4] 2> 3174078 T8855 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index/
[junit4] 2> 3174078 T8855 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3174078 T8855 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-004/collection1/data/index
[junit4] 2> 3174078 T8855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=46.5205078125, floorSegmentMB=0.3212890625, forceMergeDeletesPctAllowed=16.008820695250826, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2865870921699244
[junit4] 2> 3174079 T8855 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57452f51 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a47a5e8),segFN=segments_1,generation=1}
[junit4] 2> 3174079 T8855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3174083 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3174084 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3174085 T8855 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3174086 T8855 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3174086 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3174086 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3174087 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3174087 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3174088 T8855 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3174089 T8855 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3174090 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3174090 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3174091 T8855 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3174105 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3174107 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3174108 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3174109 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3174111 T8855 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3174115 T8855 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3174116 T8855 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3174116 T8855 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=49.939453125, floorSegmentMB=1.9423828125, forceMergeDeletesPctAllowed=2.637825278493197, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3174117 T8855 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@57452f51 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6a47a5e8),segFN=segments_1,generation=1}
[junit4] 2> 3174117 T8855 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3174117 T8855 oass.SolrIndexSearcher.<init> Opening Searcher@6ebe415e[collection1] main
[junit4] 2> 3174117 T8855 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3174118 T8855 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3174118 T8855 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3174119 T8855 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3174119 T8855 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3174119 T8855 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3174120 T8855 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3174120 T8855 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3174120 T8855 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3174120 T8855 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3174123 T8856 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ebe415e[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3174126 T8855 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3174126 T8859 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10100/_x/c collection:collection1 shard:shard2
[junit4] 2> 3174127 T8786 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 3174127 T8786 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3174127 T8859 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 3174131 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 3174132 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3174132 T8859 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3174132 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1449 name=collection1 org.apache.solr.core.SolrCore@320c550b url=http://127.0.0.1:10100/_x/c/collection1 node=127.0.0.1:10100__x%2Fc C1449_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:10100/_x/c, core=collection1, node_name=127.0.0.1:10100__x%2Fc}
[junit4] 2> 3174133 T8859 C1449 P10100 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10100/_x/c/collection1/
[junit4] 2> 3174133 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3174133 T8859 C1449 P10100 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3174134 T8859 C1449 P10100 oasc.SyncStrategy.syncToMe http://127.0.0.1:10100/_x/c/collection1/ has no replicas
[junit4] 2> 3174134 T8859 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10100/_x/c/collection1/ shard2
[junit4] 2> 3174134 T8859 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 3174135 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3174135 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174135 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174135 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174135 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174138 T8809 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3174139 T8821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3174240 T8835 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174240 T8819 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174240 T8854 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174240 T8821 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3174290 T8859 oasc.ZkController.register We are http://127.0.0.1:10100/_x/c/collection1/ and leader is http://127.0.0.1:10100/_x/c/collection1/
[junit4] 2> 3174291 T8859 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10100/_x/c
[junit4] 2> 3174291 T8859 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3174291 T8859 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3174291 T8859 oasc.ZkController.publish numSh
[...truncated too long message...]
-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data;done=false>>]
[junit4] 2> 3275946 T8786 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data
[junit4] 2> 3275947 T10321 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:30546__x%2Fc
[junit4] 2> 3275950 T10323 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 3277450 T10316 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 3277453 T8786 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_x/c,null}
[junit4] 2> 3280286 T8786 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 3280289 T8786 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37683 37683
[junit4] 2> 3280518 T8787 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37683 37683
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=9BBB3C28E81C7420 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg -Dtests.timezone=Europe/Malta -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 119s J1 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([9BBB3C28E81C7420:1A5DB2309F43141C]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3280546 T8786 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-9BBB3C28E81C7420-001
[junit4] 2> 118560 T8785 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=bg, timezone=Europe/Malta
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=231981400,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [SolrTestCaseJ4Test, TestSchemaNameResource, HdfsUnloadDistributedZkTest, TestSolr4Spatial2, TestCSVResponseWriter, TestStressLucene, TestFreeTextSuggestions, LoggingHandlerTest, TestFieldSortValues, DefaultValueUpdateProcessorTest, LeaderElectionIntegrationTest, TestConfigSets, CollectionsAPIDistributedZkTest, TestMergePolicyConfig, HdfsChaosMonkeySafeLeaderTest, AnalysisAfterCoreReloadTest, TestQueryUtils, TestExtendedDismaxParser, DirectUpdateHandlerOptimizeTest, TestCollapseQParserPlugin, UpdateRequestProcessorFactoryTest, CopyFieldTest, DocValuesMissingTest, TermVectorComponentTest, FullSolrCloudDistribCmdsTest, TestCloudSchemaless, TestSerializedLuceneMatchVersion, TestRestManager, TestSuggestSpellingConverter, TestDistributedMissingSort, TestPseudoReturnFields, ZkSolrClientTest, OpenCloseCoreStressTest, TestRandomMergePolicy, TestBinaryField, RankQueryTest, TestShardHandlerFactory, TestCollationField, TestIndexSearcher, AutoCommitTest, DistanceFunctionTest, CSVRequestHandlerTest, TestRealTimeGet, TestSolrXMLSerializer, TestLRUCache, DeleteReplicaTest, TestSolrXmlPersistor, TestReversedWildcardFilterFactory, SimplePostToolTest, DistributedQueryComponentOptimizationTest, DocValuesTest, TestCloudManagedSchema, TestHighFrequencyDictionaryFactory, UUIDUpdateProcessorFallbackTest, DistributedQueryElevationComponentTest, SynonymTokenizerTest, TestRecovery, TestNonNRTOpen, TestCursorMarkWithoutUniqueKey, IndexSchemaRuntimeFieldTest, DateMathParserTest, SOLR749Test, UniqFieldsUpdateProcessorFactoryTest, TestShortCircuitedRequests, MergeStrategyTest, SolrIndexConfigTest, SolrInfoMBeanTest, DistributedQueryComponentCustomSortTest, TestSolrQueryParserDefaultOperatorResource, TestMaxScoreQueryParser, TestGroupingSearch, DebugComponentTest, BinaryUpdateRequestHandlerTest, DistributedSuggestComponentTest, TestPostingsSolrHighlighter, CurrencyFieldXmlFileTest, TestIBSimilarityFactory, ScriptEngineTest, StressHdfsTest, TestFieldCollectionResource, TestPHPSerializedResponseWriter, FieldAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderTest, HttpPartitionTest, SampleTest, ShardRoutingCustomTest, ConvertedLegacyTest, TestDistribDocBasedVersion, TestSystemIdResolver, TestCollectionAPI, HardAutoCommitTest, SpatialFilterTest, RangeFacetTest, HdfsRecoveryZkTest, SliceStateTest, SchemaVersionSpecificBehaviorTest, CacheHeaderTest, BasicZkTest, TestSweetSpotSimilarityFactory, SolrRequestParserTest, CursorPagingTest, BasicDistributedZk2Test, UnloadDistributedZkTest, ClusterStateUpdateTest, TestRandomFaceting, TestZkChroot, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestStressVersions, DistributedTermsComponentTest, SolrCoreTest, SpellCheckComponentTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, OverseerCollectionProcessorTest, SoftAutoCommitTest, CurrencyFieldOpenExchangeTest, TestCoreDiscovery, SuggesterFSTTest, TestTrie, TestCSVLoader, NoCacheHeaderTest, TestWordDelimiterFilterFactory, DocValuesMultiTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestQueryTypes, FileBasedSpellCheckerTest, TestValueSourceCache, TestSolrQueryParser, IndexSchemaTest, JSONWriterTest, QueryParsingTest, JsonLoaderTest, TestComponentsName, SearchHandlerTest, TestQuerySenderListener, UpdateParamsTest, TestQuerySenderNoQuery, TestSolrDeletionPolicy2, MultiTermTest, NumericFieldsTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, ResourceLoaderTest, URLClassifyProcessorTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, CollectionsAPIAsyncDistributedZkTest, DeleteShardTest, DistribCursorPagingTest, MultiThreadedOCPTest]
[junit4] Completed on J1 in 118.89s, 1 test, 1 failure <<< FAILURES!
[...truncated 226 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1761 tests, 1 failure, 53 ignored (19 assumptions)
Total time: 106 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 228016853 bytes
Compression is 0.0%
Took 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4790 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4790/
2 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:FFB4043F1FA92C1]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:8A1541A894001DDC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11379 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerTest-BF3CFB0E35F7DE0-001/init-core-data-001
[junit4] 2> 1856167 T3762 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 1856168 T3762 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1856168 T3762 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1856171 T3762 oas.SolrTestCaseJ4.setUp ###Starting testShardLeaderChange
[junit4] 2> 1856171 T3762 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1856172 T3763 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1856272 T3762 oasc.ZkTestServer.run start zk server on port:18946
[junit4] 2> 1856273 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856277 T3769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@497deeb6 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856277 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856278 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856279 T3771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d172e27 name:ZooKeeperConnection Watcher:127.0.0.1:18946 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856279 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856281 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856282 T3773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b5214f6 name:ZooKeeperConnection Watcher:127.0.0.1:18946 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856282 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856282 T3762 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1856284 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1856287 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856288 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856288 T3776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b9d22c2 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856289 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856289 T3778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51a20a18 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856289 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856289 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1856289 T3762 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1856291 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1856291 T3762 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1856292 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1856294 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1856294 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1856294 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1856295 T3774 oasc.Overseer.start Overseer (id=92227472330063875-127.0.0.1:18946_solr-n_0000000000) starting
[junit4] 2> 1856297 T3762 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1856297 T3774 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1856299 T3774 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1856301 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1856303 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1856305 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1856308 T3774 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1856311 T3781 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1856311 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 1856313 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856314 T3783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46a416ea name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856314 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856315 T3781 oasc.OverseerCollectionProcessor.run ERROR Unable to prioritize overseer org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /roles.json
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:251)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:248)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:248)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.prioritizeOverseerNodes(OverseerCollectionProcessor.java:412)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:256)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1856317 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1856317 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1856319 T3774 oasc.Overseer.start Overseer (id=92227472330063877-127.0.0.1:18946_solr-n_0000000001) starting
[junit4] 2> 1856326 T3785 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1856326 T3784 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1856328 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1856328 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1856328 T3784 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 1856329 T3784 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1856330 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856331 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1856800 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1856806 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1856810 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856811 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856812 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1856811 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1856815 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1856815 T3784 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1856817 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856817 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1856818 T3788 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b928720 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1856818 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1856819 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1856821 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1856825 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856825 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1856826 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1856826 T3784 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1856828 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1856829 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1856930 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1856930 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1857031 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1857033 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857034 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857034 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857035 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4] 2> 1857036 T3784 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1857038 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857139 T3779 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1857139 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1857533 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1857539 T3792 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1857540 T3789 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1857543 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857543 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857543 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857544 T3784 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1857547 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1857649 T3789 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1859039 T3779 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1859041 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1859043 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1859044 T3794 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a27853f name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1859045 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1859046 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1859047 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1859051 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1859051 T3791 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1859051 T3790 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1859051 T3786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1859052 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 1859054 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1859055 T3797 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40ef5f7d name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1859055 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1859056 T3784 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
[junit4] 2> 1859057 T3798 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 1859058 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1859059 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1859059 T3795 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1859060 T3774 oasc.Overseer.start Overseer (id=92227472330063880-127.0.0.1:18946_solr-n_0000000002) starting
[junit4] 2> 1859067 T3800 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1859068 T3799 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1859070 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1859070 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1859071 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1859072 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1859073 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1859075 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1859077 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1859179 T3795 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1860558 T3789 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1860561 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1860561 T3803 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860561 T3802 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860561 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860563 T3799 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1860563 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1860564 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1860565 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1860566 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860567 T3805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2faf0bab name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1860567 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1860568 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1860570 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1860573 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860573 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1860574 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1860574 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1860576 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860578 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1860580 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860580 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1860581 T3799 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1860581 T3799 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1860583 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860585 T3807 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1860586 T3808 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1860586 T3794 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@39e61331 rejected from java.util.concurrent.ThreadPoolExecutor@2ff613a9[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 6]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:201)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1860590 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860592 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1860694 T3808 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1862085 T3795 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1862086 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862088 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1862090 T3811 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@196ac902 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1862090 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1862091 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1862093 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1862096 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 1862097 T3801 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862097 T3803 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862097 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1862097 T3802 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862097 T3799 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.orderedChildren(DistributedQueue.java:99)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:587)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:217)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1862098 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1862099 T3799 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
[junit4] 2> 1862100 T3815 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 1862100 T3814 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b2a743e name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1862101 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1862103 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1862104 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1862105 T3812 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1862105 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1862105 T3774 oasc.Overseer.start Overseer (id=92227472330063883-127.0.0.1:18946_solr-n_0000000003) starting
[junit4] 2> 1862110 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1862112 T3817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41c55b19 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1862112 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1862113 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1862114 T3819 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1862115 T3818 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1862115 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1862116 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862117 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1862118 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1862118 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1862118 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862120 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1862120 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1862123 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862123 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862124 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1862124 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1862125 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862128 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862128 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1862128 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1862130 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862231 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1862231 T3812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1862324 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862326 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862326 T3822 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862326 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1862326 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862328 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1862329 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1862329 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1862330 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862332 T3824 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1862332 T3820 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1862336 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862338 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1862440 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1863832 T3812 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1863834 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1863836 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1863837 T3826 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16902c1c name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1863837 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1863838 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1863840 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1863844 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1863844 T3821 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863844 T3822 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863844 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863846 T3818 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1863847 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863847 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863849 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863850 T3827 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1863851 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863851 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1863852 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863852 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863854 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863856 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1863856 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863857 T3829 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c1bc032 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1863857 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1863858 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1863860 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1863863 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863863 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1863864 T3818 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863865 T3818 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863866 T3823 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1863868 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1863931 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 1863933 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1863934 T3832 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@703b1b63 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1863934 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1863934 T3818 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
[junit4] 2> 1863936 T3833 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 1863937 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1863938 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1863939 T3774 oasc.Overseer.start Overseer (id=92227472330063887-127.0.0.1:18946_solr-n_0000000004) starting
[junit4] 2> 1863946 T3835 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1863948 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1863948 T3834 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1863949 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863949 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863950 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863950 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863952 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863952 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863953 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863953 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863955 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863955 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863957 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1863958 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1863958 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863958 T3827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1863960 T3834 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1864070 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1864072 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1864072 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1864074 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1864074 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1864074 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1864076 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1864077 T3837 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1864078 T3830 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1864081 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1864084 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1864185 T3830 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1865577 T3827 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1865579 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1865581 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1865582 T3839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e2c5422 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1865583 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1865584 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1865586 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1865590 T3836 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865590 T3841 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865590 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865590 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1865591 T3834 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1865591 T3834 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1865592 T3834 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1865593 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865596 T3840 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1865599 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865601 T3842 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1865662 T3774 oasc.OverseerTest$OverseerRestarter.run Killing overseer.
[junit4] 2> 1865664 T3774 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1865666 T3844 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b8fc314 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1865666 T3774 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1865666 T3834 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:18946_solr
[junit4] 2> 1865667 T3845 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 1865669 T3774 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:18946_solr
[junit4] 2> 1865669 T3774 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1865671 T3774 oasc.Overseer.start Overseer (id=92227472330063889-127.0.0.1:18946_solr-n_0000000005) starting
[junit4] 2> 1865678 T3847 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1865679 T3846 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1865680 T3846 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1865681 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1865681 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1865682 T3840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1865684 T3840 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1865685 T3846 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1867095 T3830 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1867098 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1867098 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867100 T3846 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1867100 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1867101 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1867102 T3762 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1867103 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867104 T3850 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64ddd902 name:ZooKeeperConnection Watcher:127.0.0.1:18946/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1867104 T3762 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1867105 T3762 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1867107 T3762 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1867110 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1867110 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867112 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state1",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1867112 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1867114 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867115 T3762 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1867117 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867117 T3762 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1867118 T3846 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"state2",
[junit4] 2> "node_name":"node2",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node2/solr/"}
[junit4] 2> 1867118 T3846 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1867120 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867122 T3852 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1867123 T3853 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1867126 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867129 T3848 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1867230 T3853 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1868622 T3840 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:285)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:282)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPool
[...truncated too long message...]
ld/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index;done=false>>]
[junit4] 2> 3338878 T8658 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001/tempDir-006/ocptest_shardsplit_shard1_replica1/data/index
[junit4] 2> 3338879 T10223 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:31309_
[junit4] 2> 3338882 T10225 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 3340381 T10218 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 3340383 T8658 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 3342887 T8658 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 3342890 T8658 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48964 48964
[junit4] 2> 3343106 T8659 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48964 48964
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=BF3CFB0E35F7DE0 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_CR -Dtests.timezone=US/East-Indiana -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 57.1s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([BF3CFB0E35F7DE0:8A1541A894001DDC]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3343129 T8658 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-BF3CFB0E35F7DE0-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_CR, timezone=US/East-Indiana
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=255704496,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestTrie, SolrCoreCheckLockOnStartupTest, TestRemoteStreaming, CurrencyFieldOpenExchangeTest, TestReplicationHandler, SolrXmlInZkTest, AnalyticsQueryTest, SuggesterWFSTTest, ZkControllerTest, FunctionTest, TestSolrXml, SolrIndexSplitterTest, SolrCmdDistributorTest, FieldFacetTest, AddBlockUpdateTest, PrimitiveFieldTypeTest, DistributedSpellCheckComponentTest, DisMaxRequestHandlerTest, TestClassNameShortening, HdfsSyncSliceTest, TestCloudManagedSchemaConcurrent, AssignTest, TestDocBasedVersionConstraints, RecoveryZkTest, PreAnalyzedFieldTest, TestManagedResource, TestTolerantSearch, PrimUtilsTest, SignatureUpdateProcessorFactoryTest, TestSort, BasicDistributedZkTest, TestComplexPhraseQParserPlugin, HdfsBasicDistributedZkTest, RegexBoostProcessorTest, TestWordDelimiterFilterFactory, TestHighlightDedupGrouping, XsltUpdateRequestHandlerTest, SimpleFacetsTest, HdfsBasicDistributedZk2Test, ExpressionTest, TestCollationFieldDocValues, TestDefaultSimilarityFactory, PeerSyncTest, TestSolrIndexConfig, DocumentBuilderTest, TestAddFieldRealTimeGet, TestManagedStopFilterFactory, SyncSliceTest, TestConfig, TestReloadAndDeleteDocs, TestNoOpRegenerator, DocExpirationUpdateProcessorFactoryTest, TestFileDictionaryLookup, HighlighterConfigTest, TestDefaultSearchFieldResource, DistributedIntervalFacetingTest, SpellingQueryConverterTest, AddSchemaFieldsUpdateProcessorFactoryTest, ChangedSchemaMergeTest, CoreAdminHandlerTest, LeaderElectionTest, HighlighterMaxOffsetTest, TestOverriddenPrefixQueryForCustomFieldType, SuggestComponentTest, ParsingFieldUpdateProcessorsTest, BadIndexSchemaTest, WordBreakSolrSpellCheckerTest, TestSolrCoreProperties, TestIntervalFaceting, QueryEqualityTest, TestRequestStatusCollectionAPI, TestManagedSchema, TestReplicationHandlerBackup, TestCoreDiscovery, DirectUpdateHandlerTest, PluginInfoTest, SpellCheckCollatorTest, HdfsCollectionsAPIDistributedZkTest, TestArbitraryIndexDir, AnalyticsMergeStrategyTest, ReturnFieldsTest, TestStressVersions, OpenExchangeRatesOrgProviderTest, IndexBasedSpellCheckerTest, ZkCLITest, ClusterStateUpdateTest, TestSchemaResource, TestHashPartitioner, TestSchemaVersionResource, ResponseLogComponentTest, OverseerTest, TestStressRecovery, TestSearchPerf, TestQuerySenderListener, TestComponentsName, DocValuesMultiTest, TestFieldTypeResource, ShardRoutingTest, OverseerStatusTest, XmlUpdateRequestHandlerTest, OverseerCollectionProcessorTest, CoreAdminCreateDiscoverTest, TestExceedMaxTermLength, BasicDistributedZk2Test, TestZkChroot, TestRandomDVFaceting, TestDistributedSearch, TestRangeQuery, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, BasicFunctionalityTest, TestLazyCores, TestBadConfig, TestIndexSearcher, HighlighterTest, ShowFileRequestHandlerTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTest, TestCSVLoader, NoCacheHeaderTest, TestSolrDeletionPolicy1, TestSurroundQueryParser, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestQueryTypes, TermsComponentTest, TestValueSourceCache, TermVectorComponentTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestJmxIntegration, JSONWriterTest, QueryParsingTest, JsonLoaderTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestLFUCache, AlternateDirectoryTest, TestMergePolicyConfig, MultiTermTest, TestDocSet, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCodecSupport, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TimeZoneUtilsTest, URLClassifyProcessorTest, TestSystemIdResolver, DateFieldTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, TestLRUCache, TestUtils, ZkNodePropsTest, FileUtilsTest, CircularListTest, TestRTGBase, TestCursorMarkWithoutUniqueKey, AbstractAnalyticsStatsTest, AbstractAnalyticsFacetTest, AliasIntegrationTest, CustomCollectionTest, DeleteInactiveReplicaTest, DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest]
[junit4] Completed on J0 in 57.11s, 1 test, 1 failure <<< FAILURES!
[...truncated 215 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1757 tests, 1 error, 1 failure, 53 ignored (19 assumptions)
Total time: 103 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 238011058 bytes
Compression is 0.0%
Took 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4789 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4789/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=17287, name=qtp1607451782-17287, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=17287, name=qtp1607451782-17287, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at __randomizedtesting.SeedInfo.seed([EBA9F5A770D74A40]:0)
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11273 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/init-core-data-001
[junit4] 2> 1272769 T15606 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 1272770 T15606 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_ssn/xx
[junit4] 2> 1272775 T15606 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1272776 T15606 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1272777 T15607 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1272877 T15606 oasc.ZkTestServer.run start zk server on port:45003
[junit4] 2> 1272879 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1272882 T15613 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a593714 name:ZooKeeperConnection Watcher:127.0.0.1:45003 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1272883 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1272883 T15606 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1272887 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1272888 T15615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ae8a460 name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1272888 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1272889 T15606 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1272891 T15606 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1272893 T15606 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1272895 T15606 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1272897 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1272897 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1272900 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1272901 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1273003 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1273004 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1273006 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1273006 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1273008 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1273009 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1273011 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1273012 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1273014 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1273014 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1273016 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1273017 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1273019 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1273019 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1273021 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1273022 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1273024 T15606 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1273024 T15606 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1273027 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1273029 T15617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@227f0925 name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1273029 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1274156 T15606 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1274160 T15606 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1274163 T15606 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:35121
[junit4] 2> 1274165 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1274165 T15606 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1274165 T15606 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002
[junit4] 2> 1274166 T15606 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/'
[junit4] 2> 1274197 T15606 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/solr.xml
[junit4] 2> 1274250 T15606 oasc.CoreContainer.<init> New CoreContainer 690984666
[junit4] 2> 1274251 T15606 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/]
[junit4] 2> 1274252 T15606 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1274252 T15606 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1274252 T15606 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1274253 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1274253 T15606 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1274253 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1274253 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1274254 T15606 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1274254 T15606 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1274256 T15606 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1274257 T15606 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1274257 T15606 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1274258 T15606 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1274258 T15606 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45003/solr
[junit4] 2> 1274258 T15606 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1274259 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1274260 T15628 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4cabb81b name:ZooKeeperConnection Watcher:127.0.0.1:45003 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1274261 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1274263 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1274264 T15630 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f116a62 name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1274265 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1274266 T15606 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1274268 T15606 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1274270 T15606 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1274272 T15606 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1274274 T15606 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1274276 T15606 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1274277 T15606 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35121__ssn%2Fxx
[junit4] 2> 1274278 T15606 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35121__ssn%2Fxx
[junit4] 2> 1274280 T15606 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1274281 T15606 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1274284 T15606 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:35121__ssn%2Fxx
[junit4] 2> 1274284 T15606 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1274285 T15606 oasc.Overseer.start Overseer (id=92218257438212100-127.0.0.1:35121__ssn%2Fxx-n_0000000000) starting
[junit4] 2> 1274287 T15606 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1274293 T15632 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1274293 T15606 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1274295 T15606 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1274296 T15606 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1274298 T15631 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1274303 T15633 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1274303 T15633 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1274304 T15633 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1274304 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1274306 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1274306 T15631 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35121/_ssn/xx",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35121__ssn%2Fxx",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1274306 T15631 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1274306 T15631 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1274308 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1274308 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1275305 T15633 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1275306 T15633 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1275307 T15633 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1275307 T15633 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1275308 T15633 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1275308 T15633 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/collection1/'
[junit4] 2> 1275309 T15633 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 1275310 T15633 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 1275311 T15633 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 1275379 T15633 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1275428 T15633 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1275529 T15633 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1275540 T15633 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1275947 T15633 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1275960 T15633 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1275963 T15633 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1275976 T15633 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1275980 T15633 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1275985 T15633 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1275985 T15633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1275987 T15633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1275987 T15633 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1275988 T15633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1275988 T15633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1275988 T15633 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1275988 T15633 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1275989 T15633 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-001/control/data/
[junit4] 2> 1275989 T15633 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d7a6f41
[junit4] 2> 1275990 T15633 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-001/control/data
[junit4] 2> 1275990 T15633 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-001/control/data/index/
[junit4] 2> 1275990 T15633 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1276001 T15633 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-001/control/data/index
[junit4] 2> 1276001 T15633 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=12.9794921875, floorSegmentMB=1.568359375, forceMergeDeletesPctAllowed=16.968928426785745, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6718346512256952
[junit4] 2> 1276005 T15633 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1577c3a4),segFN=segments_1,generation=1}
[junit4] 2> 1276005 T15633 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1276010 T15633 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1276011 T15633 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1276011 T15633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1276011 T15633 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1276011 T15633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1276012 T15633 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1276012 T15633 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1276013 T15633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1276013 T15633 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1276013 T15633 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1276014 T15633 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1276014 T15633 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1276015 T15633 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1276015 T15633 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1276016 T15633 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1276016 T15633 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1276016 T15633 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1276017 T15633 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1276017 T15633 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1276017 T15633 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1276018 T15633 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1276018 T15633 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1276037 T15633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1276039 T15633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1276041 T15633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1276043 T15633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1276045 T15633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1276051 T15633 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1276052 T15633 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1276052 T15633 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1276056 T15633 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-001 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1577c3a4),segFN=segments_1,generation=1}
[junit4] 2> 1276056 T15633 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1276056 T15633 oass.SolrIndexSearcher.<init> Opening Searcher@1d7e24bb[collection1] main
[junit4] 2> 1276057 T15633 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1276057 T15633 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1276058 T15633 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1276058 T15633 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1276058 T15633 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1276059 T15633 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1276059 T15633 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1276059 T15633 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1276060 T15633 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 1276062 T15633 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 1276062 T15633 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1276062 T15633 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1276066 T15635 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d7e24bb[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1276069 T15633 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1276070 T15638 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:35121/_ssn/xx collection:control_collection shard:shard1
[junit4] 2> 1276070 T15606 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1276071 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1276071 T15638 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1276075 T15638 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1276076 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1276077 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276078 T15640 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30c635fb name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1276078 T15638 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1276078 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1276078 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1276078 T15638 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C898 name=collection1 org.apache.solr.core.SolrCore@4b4468f6 url=https://127.0.0.1:35121/_ssn/xx/collection1 node=127.0.0.1:35121__ssn%2Fxx C898_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:35121/_ssn/xx, core=collection1, node_name=127.0.0.1:35121__ssn%2Fxx}
[junit4] 2> 1276079 T15638 C898 P35121 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:35121/_ssn/xx/collection1/
[junit4] 2> 1276080 T15606 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1276080 T15638 C898 P35121 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1276080 T15638 C898 P35121 oasc.SyncStrategy.syncToMe https://127.0.0.1:35121/_ssn/xx/collection1/ has no replicas
[junit4] 2> 1276080 T15638 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:35121/_ssn/xx/collection1/ shard1
[junit4] 2> 1276081 T15638 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1276081 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276082 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1276082 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1276082 T15606 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1276085 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276086 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1276088 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276190 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1276190 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1276238 T15638 oasc.ZkController.register We are https://127.0.0.1:35121/_ssn/xx/collection1/ and leader is https://127.0.0.1:35121/_ssn/xx/collection1/
[junit4] 2> 1276239 T15638 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:35121/_ssn/xx
[junit4] 2> 1276239 T15638 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1276239 T15638 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1276239 T15638 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1276241 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276241 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276241 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276241 T15638 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1276242 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1276243 T15631 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:35121/_ssn/xx",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:35121__ssn%2Fxx",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1276245 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1276346 T15643 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1276346 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1277208 T15606 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1277209 T15606 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1277213 T15606 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1277215 T15606 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:61512
[junit4] 2> 1277217 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1277217 T15606 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1277217 T15606 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003
[junit4] 2> 1277217 T15606 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/'
[junit4] 2> 1277249 T15606 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/solr.xml
[junit4] 2> 1277303 T15606 oasc.CoreContainer.<init> New CoreContainer 912682939
[junit4] 2> 1277303 T15606 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/]
[junit4] 2> 1277304 T15606 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1277305 T15606 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1277305 T15606 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1277305 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1277305 T15606 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1277306 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1277306 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1277306 T15606 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1277306 T15606 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1277308 T15606 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1277310 T15606 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1277310 T15606 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1277310 T15606 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1277311 T15606 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45003/solr
[junit4] 2> 1277311 T15606 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1277312 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1277313 T15654 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a9f686 name:ZooKeeperConnection Watcher:127.0.0.1:45003 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1277314 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1277316 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1277317 T15656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ef9ffe6 name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1277317 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1277322 T15606 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1278325 T15606 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61512__ssn%2Fxx
[junit4] 2> 1278326 T15606 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:61512__ssn%2Fxx
[junit4] 2> 1278337 T15658 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1278337 T15658 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1278338 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1278338 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1278338 T15658 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1278338 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1278339 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1278340 T15631 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:61512/_ssn/xx",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:61512__ssn%2Fxx",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1278340 T15631 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 1278340 T15631 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1278342 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1278343 T15642 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1278343 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1278343 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1279339 T15658 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1279340 T15658 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1279340 T15658 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1279341 T15658 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1279341 T15658 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1279342 T15658 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/'
[junit4] 2> 1279344 T15658 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 1279345 T15658 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 1279345 T15658 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 1279413 T15658 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1279462 T15658 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1279563 T15658 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1279574 T15658 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1279973 T15658 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1279986 T15658 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1279989 T15658 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1280001 T15658 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1280005 T15658 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1280010 T15658 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1280010 T15658 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1280012 T15658 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1280012 T15658 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1280013 T15658 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1280013 T15658 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1280013 T15658 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1280014 T15658 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1280014 T15658 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/data/
[junit4] 2> 1280014 T15658 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d7a6f41
[junit4] 2> 1280015 T15658 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/data
[junit4] 2> 1280015 T15658 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/data/index/
[junit4] 2> 1280016 T15658 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1280018 T15658 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-003/collection1/data/index
[junit4] 2> 1280018 T15658 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=12.9794921875, floorSegmentMB=1.568359375, forceMergeDeletesPctAllowed=16.968928426785745, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6718346512256952
[junit4] 2> 1280021 T15658 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-002 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f3f0afb),segFN=segments_1,generation=1}
[junit4] 2> 1280022 T15658 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1280028 T15658 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1280028 T15658 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1280029 T15658 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1280029 T15658 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1280029 T15658 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1280029 T15658 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1280030 T15658 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1280030 T15658 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1280030 T15658 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1280031 T15658 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1280032 T15658 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1280032 T15658 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1280033 T15658 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1280033 T15658 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1280034 T15658 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1280034 T15658 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1280035 T15658 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1280035 T15658 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1280035 T15658 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1280036 T15658 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1280037 T15658 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1280037 T15658 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1280056 T15658 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1280058 T15658 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1280060 T15658 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1280062 T15658 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1280064 T15658 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1280070 T15658 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1280070 T15658 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1280071 T15658 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1280074 T15658 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-002 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2f3f0afb),segFN=segments_1,generation=1}
[junit4] 2> 1280074 T15658 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1280075 T15658 oass.SolrIndexSearcher.<init> Opening Searcher@529e2d4c[collection1] main
[junit4] 2> 1280075 T15658 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1280076 T15658 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1280076 T15658 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1280077 T15658 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1280077 T15658 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1280077 T15658 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1280078 T15658 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1280078 T15658 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1280078 T15658 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1280079 T15658 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1280083 T15659 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@529e2d4c[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1280087 T15658 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1280088 T15662 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:61512/_ssn/xx collection:collection1 shard:shard2
[junit4] 2> 1280088 T15606 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1280088 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1280088 T15662 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1280093 T15662 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1280094 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280095 T15662 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1280095 T15662 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C899 name=collection1 org.apache.solr.core.SolrCore@189ea944 url=https://127.0.0.1:61512/_ssn/xx/collection1 node=127.0.0.1:61512__ssn%2Fxx C899_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:61512/_ssn/xx, core=collection1, node_name=127.0.0.1:61512__ssn%2Fxx}
[junit4] 2> 1280095 T15662 C899 P61512 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:61512/_ssn/xx/collection1/
[junit4] 2> 1280095 T15662 C899 P61512 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1280095 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1280096 T15662 C899 P61512 oasc.SyncStrategy.syncToMe https://127.0.0.1:61512/_ssn/xx/collection1/ has no replicas
[junit4] 2> 1280096 T15662 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:61512/_ssn/xx/collection1/ shard2
[junit4] 2> 1280096 T15662 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1280098 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280098 T15642 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280098 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280098 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280101 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1280103 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280205 T15642 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280205 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280205 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280253 T15662 oasc.ZkController.register We are https://127.0.0.1:61512/_ssn/xx/collection1/ and leader is https://127.0.0.1:61512/_ssn/xx/collection1/
[junit4] 2> 1280254 T15662 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:61512/_ssn/xx
[junit4] 2> 1280254 T15662 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1280254 T15662 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1280254 T15662 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1280256 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280256 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280256 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280257 T15662 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1280258 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1280258 T15631 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:61512/_ssn/xx",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:61512__ssn%2Fxx",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1280260 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1280362 T15643 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280362 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1280363 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1281403 T15606 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1281404 T15606 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1281407 T15606 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 1281409 T15606 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:63449
[junit4] 2> 1281412 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1281412 T15606 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1281412 T15606 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004
[junit4] 2> 1281412 T15606 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/'
[junit4] 2> 1281446 T15606 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/solr.xml
[junit4] 2> 1281502 T15606 oasc.CoreContainer.<init> New CoreContainer 2011646240
[junit4] 2> 1281502 T15606 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/]
[junit4] 2> 1281503 T15606 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1281503 T15606 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1281504 T15606 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1281504 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1281504 T15606 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1281504 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1281504 T15606 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1281505 T15606 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1281505 T15606 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1281507 T15606 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1281509 T15606 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1281509 T15606 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1281509 T15606 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1281509 T15606 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:45003/solr
[junit4] 2> 1281510 T15606 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1281510 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1281512 T15673 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77ba7389 name:ZooKeeperConnection Watcher:127.0.0.1:45003 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1281513 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1281515 T15606 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1281516 T15675 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ffe745a name:ZooKeeperConnection Watcher:127.0.0.1:45003/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1281516 T15606 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1281521 T15606 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1282523 T15606 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63449__ssn%2Fxx
[junit4] 2> 1282524 T15606 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:63449__ssn%2Fxx
[junit4] 2> 1282535 T15677 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1282535 T15677 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1282536 T15643 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1282536 T15642 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1282536 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1282536 T15677 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1282538 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1282539 T15631 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:63449/_ssn/xx",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:63449__ssn%2Fxx",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1282539 T15631 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1282539 T15631 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1282541 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1282541 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1282541 T15676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1282541 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1282541 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1283537 T15677 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1283538 T15677 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1283539 T15677 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1283539 T15677 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1283540 T15677 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1283540 T15677 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/'
[junit4] 2> 1283542 T15677 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 1283543 T15677 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 1283543 T15677 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 1283611 T15677 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1283660 T15677 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1283761 T15677 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1283772 T15677 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1284173 T15677 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1284185 T15677 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1284188 T15677 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1284200 T15677 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1284205 T15677 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1284209 T15677 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1284210 T15677 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1284210 T15677 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1284211 T15677 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1284212 T15677 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1284212 T15677 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1284212 T15677 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1284212 T15677 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1284213 T15677 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/data/
[junit4] 2> 1284213 T15677 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4d7a6f41
[junit4] 2> 1284214 T15677 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/data
[junit4] 2> 1284214 T15677 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/data/index/
[junit4] 2> 1284214 T15677 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1284217 T15677 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-004/collection1/data/index
[junit4] 2> 1284217 T15677 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=12.9794921875, floorSegmentMB=1.568359375, forceMergeDeletesPctAllowed=16.968928426785745, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6718346512256952
[junit4] 2> 1284220 T15677 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-003 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@89cb77b),segFN=segments_1,generation=1}
[junit4] 2> 1284221 T15677 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1284227 T15677 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1284227 T15677 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1284228 T15677 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1284228 T15677 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1284228 T15677 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1284228 T15677 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1284229 T15677 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1284229 T15677 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1284230 T15677 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1284230 T15677 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1284231 T15677 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1284231 T15677 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1284232 T15677 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1284233 T15677 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1284233 T15677 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1284233 T15677 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1284234 T15677 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1284234 T15677 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1284234 T15677 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1284235 T15677 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1284236 T15677 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1284236 T15677 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1284255 T15677 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1284257 T15677 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1284259 T15677 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1284261 T15677 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1284263 T15677 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1284269 T15677 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1284269 T15677 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1284270 T15677 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1284272 T15677 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(MMapDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/index-MMapDirectory-003 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@89cb77b),segFN=segments_1,generation=1}
[junit4] 2> 1284273 T15677 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1284273 T15677 oass.SolrIndexSearcher.<init> Opening Searcher@28f1e53d[collection1] main
[junit4] 2> 1284274 T15677 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1284274 T15677 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1284275 T15677 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1284275 T15677 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1284275 T15677 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1284276 T15677 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1284276 T15677 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1284277 T15677 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1284277 T15677 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1284277 T15677 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1284281 T15678 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28f1e53d[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1284285 T15677 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1284286 T15681 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:63449/_ssn/xx collection:collection1 shard:shard1
[junit4] 2> 1284286 T15606 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1284287 T15606 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1284287 T15681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1284291 T15681 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1284292 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1284293 T15681 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1284293 T15681 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C900 name=collection1 org.apache.solr.core.SolrCore@1d61b203 url=https://127.0.0.1:63449/_ssn/xx/collection1 node=127.0.0.1:63449__ssn%2Fxx C900_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:63449/_ssn/xx, core=collection1, node_name=127.0.0.1:63449__ssn%2Fxx}
[junit4] 2> 1284293 T15681 C900 P63449 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:63449/_ssn/xx/collection1/
[junit4] 2> 1284293 T15681 C900 P63449 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1284293 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1284294 T15681 C900 P63449 oasc.SyncStrategy.syncToMe https://127.0.0.1:63449/_ssn/xx/collection1/ has no replicas
[junit4] 2> 1284294 T15681 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:63449/_ssn/xx/collection1/ shard1
[junit4] 2> 1284294 T15681 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1284296 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1284296 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284296 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284296 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284296 T15676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284299 T15631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1284301 T15634 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1284402 T15634 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284402 T15657 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284402 T15641 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1284402 T15676 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[j
[...truncated too long message...]
ts-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001/tempDir-006/ocptest2_shard3_replica1/data
[junit4] 2> 1631397 T19102 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:49385__ssn%2Fxx
[junit4] 2> 1632900 T19104 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 1632900 T19083 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 1632904 T15606 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_ssn/xx,null}
[junit4] 2> 1637325 T15606 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 1637329 T15606 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:45003 45003
[junit4] 2> 1637551 T15607 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:45003 45003
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=EBA9F5A770D74A40 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=America/Toronto -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 365s J1 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=17287, name=qtp1607451782-17287, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([EBA9F5A770D74A40]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=18950, name=qtp1607451782-18950, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([EBA9F5A770D74A40]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15687, name=qtp356211356-15687, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([EBA9F5A770D74A40]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1637595 T15606 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-EBA9F5A770D74A40-001
[junit4] 2> 364832 T15605 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=hr, timezone=America/Toronto
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=152159160,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [DirectSolrConnectionTest, NoFacetTest, TestSystemIdResolver, TestSuggestSpellingConverter, OutputWriterTest, SolrCoreTest, TestStressUserVersions, TestImplicitCoreProperties, DocValuesTest, SolrPluginUtilsTest, InfoHandlerTest, DocValuesMissingTest, TestJoin, TestNRTOpen, StressHdfsTest, TestQueryUtils, UUIDUpdateProcessorFallbackTest, DistributedTermsComponentTest, DistribCursorPagingTest, DebugComponentTest, TestConfigSets, RollingRestartTest, TestZkChroot, EnumFieldTest, BasicDistributedZk2Test, DirectUpdateHandlerOptimizeTest, TestOrdValues, TestRTGBase, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestLFUCache, CopyFieldTest, CollectionsAPIDistributedZkTest, DistributedQueryComponentCustomSortTest, CacheHeaderTest, TestQueryTypes, QueryFacetTest, SynonymTokenizerTest, TestDistributedGrouping, DateRangeFieldTest, TestSerializedLuceneMatchVersion, TestCollapseQParserPlugin, DateMathParserTest, TestCSVLoader, MultiThreadedOCPTest]
[junit4] Completed on J1 in 365.12s, 1 test, 1 error <<< FAILURES!
[...truncated 864 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1756 tests, 1 error, 53 ignored (19 assumptions)
Total time: 106 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 248963974 bytes
Compression is 0.0%
Took 42 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4788 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4788/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=58612, name=qtp1292738159-58612, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=58612, name=qtp1292738159-58612, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at __randomizedtesting.SeedInfo.seed([375F71D63CFE4C1E]:0)
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11942 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/init-core-data-001
[junit4] 2> 3334949 T58528 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 3334949 T58528 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 3334954 T58528 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 3334954 T58528 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3334955 T58529 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 3335055 T58528 oasc.ZkTestServer.run start zk server on port:11080
[junit4] 2> 3335056 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3335059 T58535 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ebfbb6c name:ZooKeeperConnection Watcher:127.0.0.1:11080 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3335059 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3335059 T58528 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 3335062 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3335063 T58537 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ffd0739 name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3335063 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3335063 T58528 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 3335065 T58528 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 3335067 T58528 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 3335068 T58528 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 3335070 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3335070 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 3335073 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 3335073 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 3335174 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3335175 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3335177 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3335177 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 3335179 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3335179 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 3335181 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3335182 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 3335183 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3335184 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 3335185 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3335186 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 3335188 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3335188 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3335190 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3335190 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 3335192 T58528 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3335192 T58528 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 3335194 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3335195 T58539 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71adc996 name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3335195 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3336061 T58528 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3336065 T58528 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3336068 T58528 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11084
[junit4] 2> 3336070 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3336071 T58528 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3336071 T58528 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002
[junit4] 2> 3336071 T58528 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/'
[junit4] 2> 3336102 T58528 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/solr.xml
[junit4] 2> 3336155 T58528 oasc.CoreContainer.<init> New CoreContainer 726019860
[junit4] 2> 3336156 T58528 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/]
[junit4] 2> 3336157 T58528 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3336157 T58528 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3336157 T58528 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3336157 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3336158 T58528 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3336158 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3336158 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3336158 T58528 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3336159 T58528 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3336161 T58528 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3336162 T58528 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3336162 T58528 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3336163 T58528 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3336163 T58528 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:11080/solr
[junit4] 2> 3336163 T58528 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3336164 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3336165 T58550 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2d4c13 name:ZooKeeperConnection Watcher:127.0.0.1:11080 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3336166 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3336168 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3336169 T58552 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1feabd30 name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3336169 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3336170 T58528 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 3336172 T58528 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 3336175 T58528 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 3336177 T58528 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 3336179 T58528 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 3336182 T58528 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 3336183 T58528 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11084_
[junit4] 2> 3336184 T58528 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11084_
[junit4] 2> 3336186 T58528 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 3336188 T58528 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 3336190 T58528 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:11084_
[junit4] 2> 3336191 T58528 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 3336193 T58528 oasc.Overseer.start Overseer (id=92210758576439300-127.0.0.1:11084_-n_0000000000) starting
[junit4] 2> 3336195 T58528 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 3336202 T58554 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 3336202 T58528 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 3336204 T58528 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 3336205 T58528 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3336207 T58553 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 3336229 T58555 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 3336229 T58555 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3336230 T58555 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3336230 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3336232 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3336232 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:11084",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11084_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3336232 T58553 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 3336232 T58553 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3336234 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3336235 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3337230 T58555 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3337231 T58555 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 3337232 T58555 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3337232 T58555 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3337233 T58555 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3337233 T58555 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/collection1/'
[junit4] 2> 3337236 T58555 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 3337237 T58555 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 3337238 T58555 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 3337316 T58555 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3337371 T58555 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3337472 T58555 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3337483 T58555 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3338009 T58555 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3338025 T58555 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3338028 T58555 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3338040 T58555 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3338045 T58555 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3338049 T58555 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3338050 T58555 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3338050 T58555 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3338051 T58555 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3338051 T58555 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3338052 T58555 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3338052 T58555 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 3338052 T58555 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3338053 T58555 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-001/control/data/
[junit4] 2> 3338053 T58555 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@29c22829
[junit4] 2> 3338054 T58555 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-001/control/data
[junit4] 2> 3338054 T58555 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-001/control/data/index/
[junit4] 2> 3338054 T58555 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 3338055 T58555 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-001/control/data/index
[junit4] 2> 3338055 T58555 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=48.2392578125, floorSegmentMB=2.0185546875, forceMergeDeletesPctAllowed=20.840803204404946, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7648440300171757
[junit4] 2> 3338056 T58555 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@11fbde77 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c9f3c19),segFN=segments_1,generation=1}
[junit4] 2> 3338056 T58555 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3338060 T58555 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3338060 T58555 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3338061 T58555 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3338061 T58555 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3338061 T58555 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3338062 T58555 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3338062 T58555 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3338063 T58555 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3338063 T58555 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3338063 T58555 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3338064 T58555 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3338064 T58555 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3338065 T58555 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3338065 T58555 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3338066 T58555 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3338066 T58555 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3338066 T58555 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3338067 T58555 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3338067 T58555 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3338068 T58555 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3338068 T58555 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3338069 T58555 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3338096 T58555 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3338098 T58555 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3338101 T58555 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3338103 T58555 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3338105 T58555 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3338113 T58555 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3338113 T58555 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3338114 T58555 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=77.3642578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=22.08577605843873, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8063070487066961
[junit4] 2> 3338114 T58555 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@11fbde77 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c9f3c19),segFN=segments_1,generation=1}
[junit4] 2> 3338115 T58555 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3338115 T58555 oass.SolrIndexSearcher.<init> Opening Searcher@570baf3a[collection1] main
[junit4] 2> 3338115 T58555 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3338116 T58555 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3338117 T58555 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3338117 T58555 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3338117 T58555 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3338118 T58555 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3338118 T58555 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 3338118 T58555 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3338119 T58555 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 3338121 T58555 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 3338121 T58555 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3338121 T58555 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3338124 T58557 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@570baf3a[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3338127 T58555 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3338128 T58560 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:11084 collection:control_collection shard:shard1
[junit4] 2> 3338128 T58528 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4] 2> 3338129 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3338129 T58560 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 3338133 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3338133 T58560 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3338134 T58562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@350c5693 name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3338134 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3338135 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338135 T58528 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3338135 T58560 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3338135 T58560 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2549 name=collection1 org.apache.solr.core.SolrCore@52c2d07d url=https://127.0.0.1:11084/collection1 node=127.0.0.1:11084_ C2549_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:11084, core=collection1, node_name=127.0.0.1:11084_}
[junit4] 2> 3338136 T58560 C2549 P11084 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:11084/collection1/
[junit4] 2> 3338136 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3338136 T58560 C2549 P11084 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3338136 T58560 C2549 P11084 oasc.SyncStrategy.syncToMe https://127.0.0.1:11084/collection1/ has no replicas
[junit4] 2> 3338136 T58528 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3338137 T58560 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:11084/collection1/ shard1
[junit4] 2> 3338137 T58560 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 3338138 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338138 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3338139 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3338141 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338142 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3338144 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338245 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3338246 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3338294 T58560 oasc.ZkController.register We are https://127.0.0.1:11084/collection1/ and leader is https://127.0.0.1:11084/collection1/
[junit4] 2> 3338295 T58560 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:11084
[junit4] 2> 3338295 T58560 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3338295 T58560 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 3338295 T58560 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3338296 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338297 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338297 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338297 T58560 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3338298 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3338299 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:11084",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11084_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3338300 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3338402 T58565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3338402 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3339093 T58528 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 3339093 T58528 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3339097 T58528 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3339099 T58528 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11091
[junit4] 2> 3339101 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3339101 T58528 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3339101 T58528 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003
[junit4] 2> 3339102 T58528 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/'
[junit4] 2> 3339136 T58528 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/solr.xml
[junit4] 2> 3339208 T58528 oasc.CoreContainer.<init> New CoreContainer 886366504
[junit4] 2> 3339209 T58528 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/]
[junit4] 2> 3339210 T58528 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3339210 T58528 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3339210 T58528 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3339211 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3339211 T58528 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3339211 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3339212 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3339212 T58528 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3339212 T58528 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3339214 T58528 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3339216 T58528 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3339216 T58528 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3339217 T58528 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3339217 T58528 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:11080/solr
[junit4] 2> 3339217 T58528 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3339218 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3339219 T58576 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55db8b29 name:ZooKeeperConnection Watcher:127.0.0.1:11080 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3339220 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3339221 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3339222 T58578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63aea6a8 name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3339222 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3339227 T58528 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3340229 T58528 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11091_
[junit4] 2> 3340230 T58528 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11091_
[junit4] 2> 3340240 T58580 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3340240 T58580 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3340241 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3340241 T58580 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3340241 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3340241 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3340242 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3340243 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:11091",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11091_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3340243 T58553 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 3340243 T58553 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 3340245 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3340246 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3340246 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3340246 T58564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3341241 T58580 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3341242 T58580 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3341243 T58580 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3341243 T58580 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3341244 T58580 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3341244 T58580 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/'
[junit4] 2> 3341247 T58580 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 3341248 T58580 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 3341249 T58580 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 3341341 T58580 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3341396 T58580 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3341497 T58580 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3341527 T58580 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3341987 T58580 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3342002 T58580 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3342006 T58580 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3342018 T58580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3342023 T58580 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3342027 T58580 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3342028 T58580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3342029 T58580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3342029 T58580 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3342030 T58580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3342030 T58580 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3342030 T58580 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3342031 T58580 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3342031 T58580 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/data/
[junit4] 2> 3342031 T58580 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@29c22829
[junit4] 2> 3342032 T58580 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/data
[junit4] 2> 3342033 T58580 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/data/index/
[junit4] 2> 3342033 T58580 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3342033 T58580 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-003/collection1/data/index
[junit4] 2> 3342034 T58580 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=48.2392578125, floorSegmentMB=2.0185546875, forceMergeDeletesPctAllowed=20.840803204404946, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7648440300171757
[junit4] 2> 3342035 T58580 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@44bc0bbc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@329becd3),segFN=segments_1,generation=1}
[junit4] 2> 3342035 T58580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3342040 T58580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3342040 T58580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3342040 T58580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3342041 T58580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3342041 T58580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3342041 T58580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3342042 T58580 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3342042 T58580 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3342042 T58580 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3342043 T58580 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3342044 T58580 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3342044 T58580 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3342045 T58580 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3342046 T58580 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3342046 T58580 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3342046 T58580 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3342047 T58580 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3342047 T58580 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3342048 T58580 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3342048 T58580 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3342049 T58580 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3342049 T58580 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3342076 T58580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3342078 T58580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3342080 T58580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3342083 T58580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3342085 T58580 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3342092 T58580 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3342092 T58580 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3342093 T58580 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=77.3642578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=22.08577605843873, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8063070487066961
[junit4] 2> 3342094 T58580 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@44bc0bbc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@329becd3),segFN=segments_1,generation=1}
[junit4] 2> 3342094 T58580 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3342095 T58580 oass.SolrIndexSearcher.<init> Opening Searcher@567271f1[collection1] main
[junit4] 2> 3342095 T58580 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3342096 T58580 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3342096 T58580 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3342097 T58580 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3342097 T58580 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3342097 T58580 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3342098 T58580 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3342098 T58580 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3342099 T58580 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3342099 T58580 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3342103 T58581 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@567271f1[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3342107 T58580 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3342108 T58584 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:11091 collection:collection1 shard:shard2
[junit4] 2> 3342108 T58528 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4] 2> 3342109 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3342109 T58584 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 3342114 T58584 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 3342115 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342116 T58584 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3342116 T58584 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2550 name=collection1 org.apache.solr.core.SolrCore@7e8756ba url=https://127.0.0.1:11091/collection1 node=127.0.0.1:11091_ C2550_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:11091, core=collection1, node_name=127.0.0.1:11091_}
[junit4] 2> 3342116 T58584 C2550 P11091 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:11091/collection1/
[junit4] 2> 3342116 T58584 C2550 P11091 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3342117 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3342117 T58584 C2550 P11091 oasc.SyncStrategy.syncToMe https://127.0.0.1:11091/collection1/ has no replicas
[junit4] 2> 3342117 T58584 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:11091/collection1/ shard2
[junit4] 2> 3342117 T58584 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 3342119 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342119 T58564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342119 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342119 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342122 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3342124 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342226 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342226 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342226 T58564 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342274 T58584 oasc.ZkController.register We are https://127.0.0.1:11091/collection1/ and leader is https://127.0.0.1:11091/collection1/
[junit4] 2> 3342275 T58584 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:11091
[junit4] 2> 3342275 T58584 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3342275 T58584 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3342275 T58584 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3342276 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342276 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342276 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342277 T58584 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3342278 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3342278 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:11091",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11091_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3342280 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3342382 T58565 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342382 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3342382 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3343283 T58528 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 3343284 T58528 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3343287 T58528 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3343289 T58528 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:11097
[junit4] 2> 3343291 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3343292 T58528 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3343292 T58528 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004
[junit4] 2> 3343292 T58528 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/'
[junit4] 2> 3343322 T58528 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/solr.xml
[junit4] 2> 3343375 T58528 oasc.CoreContainer.<init> New CoreContainer 1166727898
[junit4] 2> 3343376 T58528 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/]
[junit4] 2> 3343377 T58528 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3343377 T58528 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3343377 T58528 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3343378 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3343378 T58528 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3343378 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3343378 T58528 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3343379 T58528 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3343379 T58528 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3343381 T58528 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3343382 T58528 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3343383 T58528 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3343383 T58528 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3343383 T58528 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:11080/solr
[junit4] 2> 3343383 T58528 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3343384 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3343386 T58595 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bbbfd3 name:ZooKeeperConnection Watcher:127.0.0.1:11080 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3343386 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3343388 T58528 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3343388 T58597 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a4e916b name:ZooKeeperConnection Watcher:127.0.0.1:11080/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3343389 T58528 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3343394 T58528 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3344397 T58528 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11097_
[junit4] 2> 3344398 T58528 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11097_
[junit4] 2> 3344408 T58599 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3344408 T58599 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3344409 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3344409 T58599 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3344409 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3344409 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3344411 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3344412 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:11097",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11097_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3344412 T58553 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 3344412 T58553 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3344414 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3344414 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3344414 T58598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3344414 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3344414 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3345410 T58599 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3345411 T58599 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3345412 T58599 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3345412 T58599 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3345413 T58599 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3345413 T58599 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/'
[junit4] 2> 3345415 T58599 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 3345416 T58599 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 3345417 T58599 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 3345487 T58599 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 3345538 T58599 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3345639 T58599 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3345651 T58599 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3346089 T58599 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3346103 T58599 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3346107 T58599 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3346118 T58599 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3346123 T58599 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3346127 T58599 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3346128 T58599 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3346128 T58599 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3346129 T58599 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3346129 T58599 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3346130 T58599 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3346130 T58599 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3346130 T58599 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3346131 T58599 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/data/
[junit4] 2> 3346131 T58599 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@29c22829
[junit4] 2> 3346132 T58599 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/data
[junit4] 2> 3346132 T58599 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/data/index/
[junit4] 2> 3346132 T58599 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3346133 T58599 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001/tempDir-004/collection1/data/index
[junit4] 2> 3346133 T58599 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=48.2392578125, floorSegmentMB=2.0185546875, forceMergeDeletesPctAllowed=20.840803204404946, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7648440300171757
[junit4] 2> 3346134 T58599 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3640d2e0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71bfb7f8),segFN=segments_1,generation=1}
[junit4] 2> 3346134 T58599 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3346139 T58599 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3346139 T58599 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3346140 T58599 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3346140 T58599 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3346140 T58599 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3346141 T58599 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3346141 T58599 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3346142 T58599 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3346142 T58599 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3346142 T58599 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3346143 T58599 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3346144 T58599 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3346144 T58599 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3346145 T58599 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3346146 T58599 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3346146 T58599 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3346146 T58599 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3346147 T58599 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3346147 T58599 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3346148 T58599 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3346148 T58599 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3346149 T58599 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3346170 T58599 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3346172 T58599 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3346174 T58599 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3346176 T58599 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3346179 T58599 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3346184 T58599 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3346184 T58599 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3346185 T58599 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=77.3642578125, floorSegmentMB=1.388671875, forceMergeDeletesPctAllowed=22.08577605843873, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8063070487066961
[junit4] 2> 3346186 T58599 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@3640d2e0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@71bfb7f8),segFN=segments_1,generation=1}
[junit4] 2> 3346186 T58599 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3346186 T58599 oass.SolrIndexSearcher.<init> Opening Searcher@326f7c43[collection1] main
[junit4] 2> 3346187 T58599 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3346188 T58599 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3346188 T58599 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3346188 T58599 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3346189 T58599 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3346189 T58599 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3346190 T58599 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3346190 T58599 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3346190 T58599 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3346191 T58599 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3346195 T58600 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@326f7c43[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3346199 T58599 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3346200 T58603 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:11097 collection:collection1 shard:shard1
[junit4] 2> 3346200 T58528 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0
[junit4] 2> 3346200 T58528 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3346200 T58603 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 3346205 T58603 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3346206 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346207 T58603 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3346207 T58603 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2551 name=collection1 org.apache.solr.core.SolrCore@7aff6248 url=https://127.0.0.1:11097/collection1 node=127.0.0.1:11097_ C2551_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:11097, core=collection1, node_name=127.0.0.1:11097_}
[junit4] 2> 3346207 T58603 C2551 P11097 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:11097/collection1/
[junit4] 2> 3346208 T58603 C2551 P11097 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3346208 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3346208 T58603 C2551 P11097 oasc.SyncStrategy.syncToMe https://127.0.0.1:11097/collection1/ has no replicas
[junit4] 2> 3346208 T58603 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:11097/collection1/ shard1
[junit4] 2> 3346209 T58603 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 3346210 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346211 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346211 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346211 T58598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346211 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346214 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3346216 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346318 T58563 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346318 T58579 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346318 T58598 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346318 T58556 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3346365 T58603 oasc.ZkController.register We are https://127.0.0.1:11097/collection1/ and leader is https://127.0.0.1:11097/collection1/
[junit4] 2> 3346366 T58603 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:11097
[junit4] 2> 3346366 T58603 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3346366 T58603 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3346366 T58603 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3346367 T58556 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346367 T58565 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346367 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3346368 T58603 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3346369 T58553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3346370 T58553 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:11097",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:11097_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 3346371 T58564 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on pa
[...truncated too long message...]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=375F71D63CFE4C1E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro_RO -Dtests.timezone=America/Buenos_Aires -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 244s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=58612, name=qtp1292738159-58612, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([375F71D63CFE4C1E]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=58745, name=qtp79004516-58745, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([375F71D63CFE4C1E]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3579038 T58528 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-375F71D63CFE4C1E-001
[junit4] 2> 244092 T58527 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ro_RO, timezone=America/Buenos_Aires
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=276433688,total=533725184
[junit4] 2> NOTE: All tests run in this JVM: [TestLMDirichletSimilarityFactory, TestReloadAndDeleteDocs, TestMissingGroups, TestManagedResource, TestDefaultSearchFieldResource, TestFuzzyAnalyzedSuggestions, TestStressReorder, TestReplicationHandler, HighlighterTest, AbstractAnalyticsStatsTest, ResponseLogComponentTest, HdfsWriteToMultipleCollectionsTest, TestIntervalFaceting, TestExceedMaxTermLength, TestFileDictionaryLookup, TestReRankQParserPlugin, SpellCheckComponentTest, MBeansHandlerTest, TestManagedStopFilterFactory, AbstractAnalyticsFacetTest, TestSolrXmlPersistence, ParsingFieldUpdateProcessorsTest, SolrCmdDistributorTest, TestUtils, TestSchemaVersionResource, PreAnalyzedFieldTest, TestFoldingMultitermQuery, XsltUpdateRequestHandlerTest, BadComponentTest, SuggestComponentTest, ShowFileRequestHandlerTest, CoreMergeIndexesAdminHandlerTest, AnalyticsMergeStrategyTest, SpellCheckCollatorTest, WordBreakSolrSpellCheckerTest, TestStressRecovery, HdfsCollectionsAPIDistributedZkTest, BlockCacheTest, TestStandardQParsers, FileUtilsTest, ReplicationFactorTest, TestCodecSupport, TestSchemaSimilarityResource, UpdateParamsTest, TestSearchPerf, TestManagedSchema, TestSolrJ, TestRequestStatusCollectionAPI, AssignTest, TestCloudManagedSchemaConcurrent, UnloadDistributedZkTest, OpenExchangeRatesOrgProviderTest, PrimitiveFieldTypeTest, OverseerCollectionProcessorTest, TestArbitraryIndexDir, TestStressVersions, ZkCLITest, DistributedDebugComponentTest, AnalyticsQueryTest, DirectUpdateHandlerTest, TestInitQParser, TestFaceting, DistributedIntervalFacetingTest, FieldMutatingUpdateProcessorTest, HdfsBasicDistributedZkTest, TestRangeQuery, TestCoreDiscovery, JSONWriterTest, IndexSchemaTest, TestComponentsName, TestDFRSimilarityFactory, TestReplicationHandlerBackup, AddSchemaFieldsUpdateProcessorFactoryTest, ClusterStateUpdateTest, PingRequestHandlerTest, MinimalSchemaTest, TestCollationFieldDocValues, PluginInfoTest, TestCustomSort, TestFieldResource, ChangedSchemaMergeTest, TestRandomDVFaceting, TestSort, TestSolrIndexConfig, TimeZoneUtilsTest, NoCacheHeaderTest, SimpleFacetsTest, TestCopyFieldCollectionResource, MoreLikeThisHandlerTest, OverseerStatusTest, FunctionTest, FastVectorHighlighterTest, CoreAdminRequestStatusTest, TestDynamicFieldCollectionResource, HdfsBasicDistributedZk2Test, TestSurroundQueryParser, TestQuerySenderListener, DocumentAnalysisRequestHandlerTest, ZkControllerTest, TestHighlightDedupGrouping, DirectSolrSpellCheckerTest, ReturnFieldsTest, BasicDistributedZkTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, TestRandomFaceting, TestDistributedSearch, TestRealTimeGet, TestJoin, TestCoreContainer, StatsComponentTest, TestGroupingSearch, ConvertedLegacyTest, TestFiltering, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, SolrRequestParserTest, SuggesterTSTTest, TestTrie, PolyFieldTest, SolrCoreCheckLockOnStartupTest, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, CacheHeaderTest, TestWriterPerf, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TermVectorComponentTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, UpdateRequestProcessorFactoryTest, QueryParsingTest, TestAnalyzedSuggestions, JsonLoaderTest, TestPartialUpdateDeduplication, HighlighterConfigTest, TestSolrDeletionPolicy2, MultiTermTest, TestConfig, ExternalFileFieldSortTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, URLClassifyProcessorTest, TestSolrXMLSerializer, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, ClusterStateTest, TestDocumentBuilder, ZkNodePropsTest, SystemInfoHandlerTest, CircularListTest, TestRTGBase, TestDistributedMissingSort, TestTolerantSearch, NoFacetTest, ExpressionTest, FieldFacetTest, RangeFacetTest, AliasIntegrationTest, ConnectionManagerTest, CustomCollectionTest, DeleteReplicaTest, DistribCursorPagingTest, MultiThreadedOCPTest]
[junit4] Completed on J0 in 244.87s, 1 test, 1 error <<< FAILURES!
[...truncated 190 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1756 tests, 1 error, 53 ignored (19 assumptions)
Total time: 113 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 199505667 bytes
Compression is 0.0%
Took 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4787 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4787/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.URL.openStream(URL.java:1037)
at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
at __randomizedtesting.SeedInfo.seed([CF5BC0D707786548]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup] at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at java.net.Socket.connect(Socket.java:528) at sun.net.NetworkClient.doConnect(NetworkClient.java:180) at sun.net.www.http.HttpClient.openServer(HttpClient.java:432) at sun.net.www.http.HttpClient.openServer(HttpClient.java:527) at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652) at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323) at java.net.URL.openStream(URL.java:1037) at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
at java.net.URL.openStream(URL.java:1037)
at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
at __randomizedtesting.SeedInfo.seed([CF5BC0D707786548]:0)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([CF5BC0D707786548:4EBD4ECF70270574]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11309 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/init-core-data-001
[junit4] 2> 1059376 T2540 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 1059377 T2540 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 1059381 T2540 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1059382 T2540 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1059383 T2541 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1059483 T2540 oasc.ZkTestServer.run start zk server on port:55768
[junit4] 2> 1059484 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1059487 T2547 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7863097 name:ZooKeeperConnection Watcher:127.0.0.1:55768 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1059488 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1059488 T2540 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1059492 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1059493 T2549 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bfea9f8 name:ZooKeeperConnection Watcher:127.0.0.1:55768/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1059493 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1059493 T2540 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1059496 T2540 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1059498 T2540 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1059499 T2540 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1059501 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1059502 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1059505 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1059506 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1059508 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1059509 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1059511 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1059512 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1059514 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1059515 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1059517 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1059518 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1059520 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1059521 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1059523 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1059523 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1059526 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1059526 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1059528 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1059529 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1059531 T2540 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1059532 T2540 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 1060745 T2540 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1060747 T2540 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14090
[junit4] 2> 1060748 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1060748 T2540 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1060748 T2540 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002
[junit4] 2> 1060749 T2540 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/'
[junit4] 2> 1060777 T2540 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/solr.xml
[junit4] 2> 1060830 T2540 oasc.CoreContainer.<init> New CoreContainer 1818329218
[junit4] 2> 1060831 T2540 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/]
[junit4] 2> 1060832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1060832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1060832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1060832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1060833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1060833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1060833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1060833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1060834 T2540 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1060834 T2540 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1060834 T2540 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1060835 T2540 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1060835 T2540 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1060835 T2540 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55768/solr
[junit4] 2> 1060835 T2540 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1060836 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1060838 T2560 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@753c0e64 name:ZooKeeperConnection Watcher:127.0.0.1:55768 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1060839 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1060840 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1060842 T2562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51c19af7 name:ZooKeeperConnection Watcher:127.0.0.1:55768/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1060842 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1060843 T2540 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1060846 T2540 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1060848 T2540 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1060850 T2540 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1060852 T2540 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1060855 T2540 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1060856 T2540 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14090_
[junit4] 2> 1060856 T2540 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14090_
[junit4] 2> 1060859 T2540 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1060860 T2540 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1060863 T2540 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:14090_
[junit4] 2> 1060863 T2540 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1060865 T2540 oasc.Overseer.start Overseer (id=92205770643144707-127.0.0.1:14090_-n_0000000000) starting
[junit4] 2> 1060867 T2540 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1060874 T2564 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1060874 T2540 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1060876 T2540 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1060877 T2540 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1060879 T2563 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1060882 T2565 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 1060883 T2565 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1060884 T2565 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1060884 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1060885 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1060885 T2563 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:14090",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:14090_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1060886 T2563 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 1060886 T2563 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1060888 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1060888 T2566 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1061884 T2565 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1061885 T2565 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 1061886 T2565 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1061886 T2565 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1061887 T2565 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1061887 T2565 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/collection1/'
[junit4] 2> 1061889 T2565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 1061890 T2565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 1061891 T2565 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 1061956 T2565 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1062005 T2565 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1062006 T2565 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1062018 T2565 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1062409 T2565 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1062422 T2565 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1062425 T2565 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1062438 T2565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1062442 T2565 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1062446 T2565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1062447 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1062448 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1062448 T2565 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1062449 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1062450 T2565 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1062450 T2565 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1062450 T2565 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1062450 T2565 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-001/control/data/
[junit4] 2> 1062451 T2565 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e748cc0
[junit4] 2> 1062452 T2565 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-001/control/data
[junit4] 2> 1062452 T2565 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-001/control/data/index/
[junit4] 2> 1062452 T2565 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 1062453 T2565 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-001/control/data/index
[junit4] 2> 1062453 T2565 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.9365234375, floorSegmentMB=1.7421875, forceMergeDeletesPctAllowed=28.541190687524374, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6084137097166367
[junit4] 2> 1062454 T2565 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@d04a904 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6342165),segFN=segments_1,generation=1}
[junit4] 2> 1062454 T2565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1062458 T2565 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1062459 T2565 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1062459 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1062460 T2565 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1062460 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1062460 T2565 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1062461 T2565 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1062461 T2565 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1062461 T2565 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1062462 T2565 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1062462 T2565 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1062462 T2565 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1062463 T2565 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1062463 T2565 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1062464 T2565 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1062464 T2565 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1062465 T2565 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1062482 T2565 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1062488 T2565 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1062488 T2565 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1062489 T2565 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=66.4814453125, floorSegmentMB=0.484375, forceMergeDeletesPctAllowed=17.16838979490969, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1062489 T2565 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@d04a904 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6342165),segFN=segments_1,generation=1}
[junit4] 2> 1062490 T2565 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1062490 T2565 oass.SolrIndexSearcher.<init> Opening Searcher@3643acf8[collection1] main
[junit4] 2> 1062490 T2565 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 1062491 T2565 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1062491 T2565 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1062492 T2565 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1062492 T2565 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1062492 T2565 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1062493 T2565 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1062493 T2565 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1062494 T2565 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 1062496 T2565 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 1062496 T2565 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1062496 T2565 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1062500 T2567 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3643acf8[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1062501 T2565 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1062502 T2570 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14090 collection:control_collection shard:shard1
[junit4] 2> 1062502 T2540 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1062503 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1062503 T2570 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 1062504 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1062506 T2572 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@225b87e6 name:ZooKeeperConnection Watcher:127.0.0.1:55768/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1062506 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1062507 T2540 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1062509 T2570 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1062510 T2540 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1062510 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062511 T2570 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1062511 T2570 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C697 name=collection1 org.apache.solr.core.SolrCore@604d3970 url=http://127.0.0.1:14090/collection1 node=127.0.0.1:14090_ C697_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:14090, core=collection1, node_name=127.0.0.1:14090_}
[junit4] 2> 1062511 T2570 C697 P14090 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:14090/collection1/
[junit4] 2> 1062512 T2570 C697 P14090 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1062512 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1062512 T2570 C697 P14090 oasc.SyncStrategy.syncToMe http://127.0.0.1:14090/collection1/ has no replicas
[junit4] 2> 1062512 T2570 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:14090/collection1/ shard1
[junit4] 2> 1062512 T2570 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 1062514 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062515 T2566 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1062515 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1062518 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1062520 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062621 T2566 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1062621 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1062670 T2570 oasc.ZkController.register We are http://127.0.0.1:14090/collection1/ and leader is http://127.0.0.1:14090/collection1/
[junit4] 2> 1062671 T2570 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14090
[junit4] 2> 1062671 T2570 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1062671 T2570 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 1062671 T2570 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1062673 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062673 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062673 T2570 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1062673 T2575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062674 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1062675 T2563 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:14090",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:14090_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1062677 T2575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1062778 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1062778 T2575 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1063730 T2540 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1063731 T2540 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1063733 T2540 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:41393
[junit4] 2> 1063734 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1063734 T2540 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1063734 T2540 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003
[junit4] 2> 1063734 T2540 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/'
[junit4] 2> 1063763 T2540 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/solr.xml
[junit4] 2> 1063829 T2540 oasc.CoreContainer.<init> New CoreContainer 503583780
[junit4] 2> 1063829 T2540 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/]
[junit4] 2> 1063831 T2540 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1063831 T2540 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1063831 T2540 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1063832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1063832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1063832 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1063833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1063833 T2540 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1063834 T2540 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1063834 T2540 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1063835 T2540 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1063835 T2540 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1063835 T2540 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1063836 T2540 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55768/solr
[junit4] 2> 1063836 T2540 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1063837 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1063839 T2586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc656e1 name:ZooKeeperConnection Watcher:127.0.0.1:55768 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1063839 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1063841 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1063842 T2588 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51a5c70 name:ZooKeeperConnection Watcher:127.0.0.1:55768/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1063842 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1063847 T2540 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1064850 T2540 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41393_
[junit4] 2> 1064851 T2540 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41393_
[junit4] 2> 1064861 T2590 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1064861 T2590 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1064862 T2575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1064862 T2590 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1064862 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1064862 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1064864 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1064865 T2563 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:41393",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41393_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1064865 T2563 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 1064866 T2563 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1064868 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1064868 T2574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1064868 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1064868 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1065863 T2590 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1065864 T2590 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1065864 T2590 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1065865 T2590 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1065865 T2590 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1065866 T2590 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/'
[junit4] 2> 1065868 T2590 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 1065869 T2590 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 1065870 T2590 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 1065933 T2590 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1065981 T2590 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1065982 T2590 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1065993 T2590 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1066373 T2590 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1066386 T2590 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1066389 T2590 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1066402 T2590 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1066406 T2590 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1066411 T2590 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1066412 T2590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1066412 T2590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1066413 T2590 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1066414 T2590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1066414 T2590 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1066414 T2590 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1066414 T2590 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1066415 T2590 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/data/
[junit4] 2> 1066415 T2590 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e748cc0
[junit4] 2> 1066416 T2590 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/data
[junit4] 2> 1066416 T2590 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/data/index/
[junit4] 2> 1066416 T2590 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1066417 T2590 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-003/collection1/data/index
[junit4] 2> 1066417 T2590 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.9365234375, floorSegmentMB=1.7421875, forceMergeDeletesPctAllowed=28.541190687524374, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6084137097166367
[junit4] 2> 1066418 T2590 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@58862a45 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@856d78c),segFN=segments_1,generation=1}
[junit4] 2> 1066418 T2590 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1066424 T2590 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1066424 T2590 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1066425 T2590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1066425 T2590 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1066425 T2590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1066425 T2590 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1066426 T2590 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1066426 T2590 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1066426 T2590 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1066427 T2590 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1066428 T2590 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1066428 T2590 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1066428 T2590 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1066429 T2590 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1066429 T2590 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1066430 T2590 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1066431 T2590 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1066449 T2590 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1066455 T2590 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1066455 T2590 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1066456 T2590 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=66.4814453125, floorSegmentMB=0.484375, forceMergeDeletesPctAllowed=17.16838979490969, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1066456 T2590 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@58862a45 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@856d78c),segFN=segments_1,generation=1}
[junit4] 2> 1066456 T2590 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1066457 T2590 oass.SolrIndexSearcher.<init> Opening Searcher@68562745[collection1] main
[junit4] 2> 1066457 T2590 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1066458 T2590 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1066458 T2590 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1066459 T2590 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1066459 T2590 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1066459 T2590 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1066460 T2590 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1066460 T2590 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1066461 T2590 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1066461 T2590 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1066465 T2591 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68562745[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1066468 T2590 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1066469 T2594 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41393 collection:collection1 shard:shard1
[junit4] 2> 1066469 T2540 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1066470 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1066470 T2594 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1066475 T2594 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1066476 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066477 T2594 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1066477 T2594 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C698 name=collection1 org.apache.solr.core.SolrCore@c940331 url=http://127.0.0.1:41393/collection1 node=127.0.0.1:41393_ C698_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:41393, core=collection1, node_name=127.0.0.1:41393_}
[junit4] 2> 1066477 T2594 C698 P41393 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41393/collection1/
[junit4] 2> 1066477 T2594 C698 P41393 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1066478 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1066478 T2594 C698 P41393 oasc.SyncStrategy.syncToMe http://127.0.0.1:41393/collection1/ has no replicas
[junit4] 2> 1066478 T2594 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41393/collection1/ shard1
[junit4] 2> 1066478 T2594 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1066480 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066481 T2574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066481 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066481 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066484 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1066486 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066587 T2574 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066588 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066587 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066636 T2594 oasc.ZkController.register We are http://127.0.0.1:41393/collection1/ and leader is http://127.0.0.1:41393/collection1/
[junit4] 2> 1066637 T2594 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41393
[junit4] 2> 1066637 T2594 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 1066637 T2594 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 1066637 T2594 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1066639 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066639 T2575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066639 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066639 T2594 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1066640 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1066641 T2563 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:41393",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:41393_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 1066643 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1066745 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066745 T2566 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1066745 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 1067722 T2540 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1067723 T2540 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1067726 T2540 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:47231
[junit4] 2> 1067726 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1067726 T2540 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1067727 T2540 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004
[junit4] 2> 1067727 T2540 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/'
[junit4] 2> 1067756 T2540 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/solr.xml
[junit4] 2> 1067809 T2540 oasc.CoreContainer.<init> New CoreContainer 1250832544
[junit4] 2> 1067809 T2540 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/]
[junit4] 2> 1067810 T2540 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 1067810 T2540 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 1067811 T2540 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1067811 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1067811 T2540 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1067811 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1067812 T2540 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1067812 T2540 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1067812 T2540 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1067813 T2540 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 1067813 T2540 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1067813 T2540 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1067813 T2540 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1067814 T2540 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:55768/solr
[junit4] 2> 1067814 T2540 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1067815 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1067816 T2605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d1a7e9d name:ZooKeeperConnection Watcher:127.0.0.1:55768 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1067817 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1067819 T2540 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1067820 T2607 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8d55316 name:ZooKeeperConnection Watcher:127.0.0.1:55768/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1067820 T2540 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1067825 T2540 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1068828 T2540 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47231_
[junit4] 2> 1068829 T2540 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47231_
[junit4] 2> 1068840 T2609 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 1068840 T2609 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 1068841 T2609 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1068841 T2574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1068841 T2575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1068841 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1068842 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1068843 T2563 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47231",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47231_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 1068843 T2563 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 1068844 T2563 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1068846 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1068846 T2573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1068846 T2566 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1068846 T2608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1068846 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1069841 T2609 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 1069842 T2609 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 1069842 T2609 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1069843 T2609 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1069844 T2609 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1069844 T2609 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/'
[junit4] 2> 1069845 T2609 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 1069846 T2609 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 1069847 T2609 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 1069911 T2609 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 1069958 T2609 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1069960 T2609 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 1069971 T2609 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 1070356 T2609 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1070369 T2609 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1070372 T2609 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1070385 T2609 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1070389 T2609 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1070393 T2609 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1070394 T2609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1070395 T2609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1070395 T2609 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1070396 T2609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1070396 T2609 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1070397 T2609 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1070397 T2609 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1070397 T2609 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/data/
[junit4] 2> 1070398 T2609 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e748cc0
[junit4] 2> 1070399 T2609 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/data
[junit4] 2> 1070399 T2609 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/data/index/
[junit4] 2> 1070399 T2609 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1070399 T2609 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-CF5BC0D707786548-001/tempDir-004/collection1/data/index
[junit4] 2> 1070400 T2609 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=48, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.9365234375, floorSegmentMB=1.7421875, forceMergeDeletesPctAllowed=28.541190687524374, segmentsPerTier=31.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6084137097166367
[junit4] 2> 1070401 T2609 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46881706 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2368d32a),segFN=segments_1,generation=1}
[junit4] 2> 1070401 T2609 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1070406 T2609 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1070407 T2609 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1070407 T2609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1070407 T2609 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1070408 T2609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1070408 T2609 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1070408 T2609 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1070409 T2609 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1070409 T2609 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1070410 T2609 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1070410 T2609 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 1070410 T2609 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 1070411 T2609 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 1070411 T2609 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 1070412 T2609 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 1070412 T2609 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1070413 T2609 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1070431 T2609 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1070436 T2609 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1070437 T2609 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1070437 T2609 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=66.4814453125, floorSegmentMB=0.484375, forceMergeDeletesPctAllowed=17.16838979490969, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1070438 T2609 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@46881706 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2368d32a),segFN=segments_1,generation=1}
[junit4] 2> 1070438 T2609 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1070439 T2609 oass.SolrIndexSearcher.<init> Opening Searcher@22c7f771[collection1] main
[junit4] 2> 1070439 T2609 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 1070440 T2609 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1070440 T2609 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1070441 T2609 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1070441 T2609 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1070441 T2609 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1070442 T2609 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 1070442 T2609 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1070443 T2609 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 1070443 T2609 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1070447 T2610 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22c7f771[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 1070450 T2609 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 1070451 T2613 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47231 collection:collection1 shard:shard2
[junit4] 2> 1070451 T2540 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 1070452 T2540 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1070452 T2613 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1070456 T2613 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 1070458 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1070458 T2613 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 1070458 T2613 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C699 name=collection1 org.apache.solr.core.SolrCore@61bfa525 url=http://127.0.0.1:47231/collection1 node=127.0.0.1:47231_ C699_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47231, core=collection1, node_name=127.0.0.1:47231_}
[junit4] 2> 1070458 T2613 C699 P47231 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47231/collection1/
[junit4] 2> 1070459 T2613 C699 P47231 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 1070459 T2613 C699 P47231 oasc.SyncStrategy.syncToMe http://127.0.0.1:47231/collection1/ has no replicas
[junit4] 2> 1070459 T2563 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 1070459 T2613 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47231/collection1/ shard2
[junit4] 2> 1070460 T2613 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1070462 T2566 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1070462 T2589 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 1070462 T2608 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 10
[...truncated too long message...]
at java.net.URL.openStream(URL.java:1037)
[junit4] 2> at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=es_US, timezone=Pacific/Nauru
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=2,free=130070176,total=497025024
[junit4] 2> NOTE: All tests run in this JVM: [ExpressionTest, TestLeaderElectionZkExpiry, TestCodecSupport, BlockCacheTest, TestLuceneMatchVersion, SignatureUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, SpellCheckCollatorTest, SuggesterTSTTest, TestCoreDiscovery, TestComplexPhraseQParserPlugin, TestSchemaSimilarityResource, DocumentBuilderTest, MBeansHandlerTest, TestFieldTypeCollectionResource, MigrateRouteKeyTest, TestInfoStreamLogging, TestOverriddenPrefixQueryForCustomFieldType, AddSchemaFieldsUpdateProcessorFactoryTest, TestDocSet, QueryResultKeyTest, TestSolrJ, TestSort, TestLMJelinekMercerSimilarityFactory, TestSearcherReuse, DocExpirationUpdateProcessorFactoryTest, TestManagedResource, SuggesterWFSTTest, AnalysisErrorHandlingTest, TestConfig, SolrCoreCheckLockOnStartupTest, ShowFileRequestHandlerTest, PluginInfoTest, XsltUpdateRequestHandlerTest, TestStressVersions, FunctionTest, TestFastLRUCache, ShardRoutingTest, TestSolrDeletionPolicy2, ZkCLITest, AnalyticsMergeStrategyTest, HdfsBasicDistributedZk2Test, TestRemoteStreaming, TestReRankQParserPlugin, SolrCmdDistributorTest, SuggesterTest, RecoveryZkTest, TestPartialUpdateDeduplication, TestWordDelimiterFilterFactory, IndexBasedSpellCheckerTest, TestFileDictionaryLookup, CustomCollectionTest, TestCustomSort, LeaderElectionTest, TestCollationFieldDocValues, TestFaceting, TestDocBasedVersionConstraints, WordBreakSolrSpellCheckerTest, OverseerTest, HdfsCollectionsAPIDistributedZkTest, TestManagedStopFilterFactory, TestDFRSimilarityFactory, TestManagedSchema, TestAnalyzedSuggestions, OverseerCollectionProcessorTest, SolrIndexSplitterTest, ChaosMonkeyNothingIsSafeTest, TestClassNameShortening, TestUtils, SpellingQueryConverterTest, OpenExchangeRatesOrgProviderTest, TestCloudInspectUtil, CoreAdminCreateDiscoverTest, TestRequestStatusCollectionAPI, TermsComponentTest, HdfsBasicDistributedZkTest, TestReplicationHandler, TestTrie, SyncSliceTest, TestDynamicFieldCollectionResource, ChangedSchemaMergeTest, UpdateParamsTest, TestManagedResourceStorage, BasicDistributedZkTest, HdfsSyncSliceTest, TestAnalyzeInfixSuggestions, DistribDocExpirationUpdateProcessorTest, TestStressRecovery, TestSurroundQueryParser, PreAnalyzedUpdateProcessorTest, DirectSolrSpellCheckerTest, QueryEqualityTest, MoreLikeThisHandlerTest, SSLMigrationTest, ExternalFileFieldSortTest, CurrencyFieldOpenExchangeTest, ResponseHeaderTest, ResponseLogComponentTest, TestRangeQuery, TestJmxIntegration, SpellCheckComponentTest, PeerSyncTest, TestSolr4Spatial, CollectionsAPIDistributedZkTest, TestZkChroot, TestRandomDVFaceting, TestDistributedGrouping, TestHashPartitioner, ZkControllerTest, TestStressReorder, TestReload, DistributedTermsComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DistributedQueryElevationComponentTest, SpatialFilterTest, NoCacheHeaderTest, DefaultValueUpdateProcessorTest, SolrInfoMBeanTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestReversedWildcardFilterFactory, XmlUpdateRequestHandlerTest, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, IndexSchemaTest, UpdateRequestProcessorFactoryTest, QueryParsingTest, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, TestComponentsName, TestLFUCache, SOLR749Test, TestSolrIndexConfig, SampleTest, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, OutputWriterTest, TestSolrCoreProperties, TestDefaultSimilarityFactory, LegacyHTMLStripCharFilterTest, TimeZoneUtilsTest, TestFastOutputStream, DateMathParserTest, PreAnalyzedFieldTest, DOMUtilTest, RAMDirectoryFactoryTest, TestLRUCache, ZkNodePropsTest, SystemInfoHandlerTest, FileUtilsTest, TestRTGBase, DistributedIntervalFacetingTest, TestTolerantSearch, AbstractAnalyticsFacetTest, FieldFacetTest, DeleteInactiveReplicaTest, HttpPartitionTest, OverseerRolesTest, ReplicationFactorTest, ShardSplitTest, HdfsRecoveryZkTest, HdfsWriteToMultipleCollectionsTest, StressHdfsTest, TestConfigSets, TestImplicitCoreProperties, TestNRTOpen, TestNonNRTOpen, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrXml, TestSolrXmlPersistence, TestReplicationHandlerBackup]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=CF5BC0D707786548 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=Pacific/Nauru -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestReplicationHandlerBackup (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:
[junit4] > 1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at java.net.Socket.connect(Socket.java:528)
[junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] > at java.net.URL.openStream(URL.java:1037)
[junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
[junit4] > at __randomizedtesting.SeedInfo.seed([CF5BC0D707786548]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=13372, name=Thread-4288, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
[junit4] > at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4] > at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
[junit4] > at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
[junit4] > at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
[junit4] > at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
[junit4] > at java.net.Socket.connect(Socket.java:579)
[junit4] > at java.net.Socket.connect(Socket.java:528)
[junit4] > at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
[junit4] > at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
[junit4] > at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
[junit4] > at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
[junit4] > at java.net.URL.openStream(URL.java:1037)
[junit4] > at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
[junit4] > at __randomizedtesting.SeedInfo.seed([CF5BC0D707786548]:0)
[junit4] Completed on J0 in 30.34s, 1 test, 2 errors <<< FAILURES!
[...truncated 162 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1754 tests, 2 suite-level errors, 1 failure, 283 ignored (19 assumptions)
Total time: 100 minutes 54 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 215875055 bytes
Compression is 0.0%
Took 38 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4786 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4786/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([B2526CA8AF3CA5D1:33B4E2B0D863C5ED]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11279 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/init-core-data-001
[junit4] 2> 936325 T7678 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 936326 T7678 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 936330 T7678 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 936331 T7678 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 936331 T7679 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 936431 T7678 oasc.ZkTestServer.run start zk server on port:10388
[junit4] 2> 936432 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 936434 T7685 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@605145a name:ZooKeeperConnection Watcher:127.0.0.1:10388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 936435 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 936435 T7678 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 936438 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 936439 T7687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@397297e name:ZooKeeperConnection Watcher:127.0.0.1:10388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 936439 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 936439 T7678 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 936441 T7678 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 936442 T7678 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 936443 T7678 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 936445 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 936446 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 936448 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 936448 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 936550 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 936551 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 936552 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 936553 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 936555 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 936555 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 936557 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 936558 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 936559 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 936560 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 936562 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 936562 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 936564 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 936565 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 936566 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 936567 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 936569 T7678 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 936569 T7678 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 937860 T7678 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 937864 T7678 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10394
[junit4] 2> 937864 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 937864 T7678 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 937865 T7678 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002
[junit4] 2> 937865 T7678 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/'
[junit4] 2> 937897 T7678 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/solr.xml
[junit4] 2> 937952 T7678 oasc.CoreContainer.<init> New CoreContainer 687547294
[junit4] 2> 937952 T7678 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/]
[junit4] 2> 937953 T7678 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 937953 T7678 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 937954 T7678 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 937954 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 937954 T7678 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 937954 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 937955 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 937955 T7678 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 937955 T7678 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 937956 T7678 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 937956 T7678 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 937956 T7678 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 937956 T7678 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 937957 T7678 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10388/solr
[junit4] 2> 937957 T7678 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 937958 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 937959 T7698 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7fdf6f16 name:ZooKeeperConnection Watcher:127.0.0.1:10388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 937960 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 937962 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 937963 T7700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5404ee5 name:ZooKeeperConnection Watcher:127.0.0.1:10388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 937963 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 937965 T7678 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 937967 T7678 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 937969 T7678 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 937971 T7678 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 937973 T7678 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 937976 T7678 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 937977 T7678 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10394_
[junit4] 2> 937977 T7678 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10394_
[junit4] 2> 937979 T7678 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 937981 T7678 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 937983 T7678 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:10394_
[junit4] 2> 937984 T7678 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 937985 T7678 oasc.Overseer.start Overseer (id=92199711310217219-127.0.0.1:10394_-n_0000000000) starting
[junit4] 2> 937987 T7678 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 937993 T7702 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 937994 T7678 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 937995 T7678 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 937997 T7678 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 937998 T7701 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 938003 T7703 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 938004 T7703 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 938005 T7703 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 938005 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 938006 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 938006 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:10394",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10394_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 938006 T7701 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 938007 T7701 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 938008 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 938009 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939005 T7703 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 939006 T7703 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 939007 T7703 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 939007 T7703 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 939007 T7703 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 939008 T7703 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/collection1/'
[junit4] 2> 939009 T7703 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 939010 T7703 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 939011 T7703 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 939078 T7703 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 939126 T7703 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 939228 T7703 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 939239 T7703 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 939620 T7703 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 939634 T7703 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 939637 T7703 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 939649 T7703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 939653 T7703 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 939657 T7703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 939658 T7703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 939659 T7703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 939660 T7703 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 939660 T7703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 939661 T7703 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 939661 T7703 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 939661 T7703 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 939661 T7703 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-001/control/data/
[junit4] 2> 939662 T7703 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@185af219
[junit4] 2> 939663 T7703 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-001/control/data
[junit4] 2> 939663 T7703 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-001/control/data/index/
[junit4] 2> 939663 T7703 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 939663 T7703 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-001/control/data/index
[junit4] 2> 939664 T7703 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1809299489, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 939665 T7703 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7b34a30e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7945e4c9),segFN=segments_1,generation=1}
[junit4] 2> 939665 T7703 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 939670 T7703 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 939670 T7703 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 939671 T7703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 939671 T7703 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 939671 T7703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 939671 T7703 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 939672 T7703 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 939672 T7703 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 939673 T7703 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 939673 T7703 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 939674 T7703 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 939674 T7703 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 939674 T7703 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 939675 T7703 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 939675 T7703 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 939676 T7703 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 939676 T7703 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 939695 T7703 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 939701 T7703 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 939701 T7703 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 939702 T7703 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=31, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 939702 T7703 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@7b34a30e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7945e4c9),segFN=segments_1,generation=1}
[junit4] 2> 939703 T7703 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 939703 T7703 oass.SolrIndexSearcher.<init> Opening Searcher@612f6d9d[collection1] main
[junit4] 2> 939703 T7703 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 939704 T7703 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 939704 T7703 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 939705 T7703 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 939705 T7703 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 939705 T7703 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 939706 T7703 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 939706 T7703 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 939707 T7703 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 939709 T7703 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 939709 T7703 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 939709 T7703 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 939712 T7705 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@612f6d9d[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 939714 T7703 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 939714 T7708 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10394 collection:control_collection shard:shard1
[junit4] 2> 939715 T7678 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 939715 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 939715 T7708 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 939717 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 939718 T7710 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@206fed2d name:ZooKeeperConnection Watcher:127.0.0.1:10388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 939718 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 939719 T7678 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 939721 T7708 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 939721 T7678 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 939722 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939722 T7708 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 939723 T7708 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C425 name=collection1 org.apache.solr.core.SolrCore@311b710c url=http://127.0.0.1:10394/collection1 node=127.0.0.1:10394_ C425_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:10394, core=collection1, node_name=127.0.0.1:10394_}
[junit4] 2> 939723 T7708 C425 P10394 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10394/collection1/
[junit4] 2> 939723 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 939723 T7708 C425 P10394 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 939724 T7708 C425 P10394 oasc.SyncStrategy.syncToMe http://127.0.0.1:10394/collection1/ has no replicas
[junit4] 2> 939724 T7708 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10394/collection1/ shard1
[junit4] 2> 939724 T7708 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 939725 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939726 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939726 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939729 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 939731 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939832 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939832 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939881 T7708 oasc.ZkController.register We are http://127.0.0.1:10394/collection1/ and leader is http://127.0.0.1:10394/collection1/
[junit4] 2> 939882 T7708 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10394
[junit4] 2> 939882 T7708 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 939882 T7708 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 939882 T7708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 939884 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939884 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939884 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939884 T7708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 939886 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 939886 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:10394",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10394_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 939888 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 939990 T7713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 939990 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 941249 T7678 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 941250 T7678 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 941253 T7678 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10401
[junit4] 2> 941253 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 941253 T7678 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 941254 T7678 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003
[junit4] 2> 941254 T7678 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/'
[junit4] 2> 941299 T7678 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/solr.xml
[junit4] 2> 941372 T7678 oasc.CoreContainer.<init> New CoreContainer 385269860
[junit4] 2> 941373 T7678 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/]
[junit4] 2> 941374 T7678 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 941374 T7678 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 941374 T7678 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 941375 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 941375 T7678 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 941375 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 941375 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 941376 T7678 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 941376 T7678 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 941376 T7678 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 941377 T7678 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 941377 T7678 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 941377 T7678 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 941377 T7678 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10388/solr
[junit4] 2> 941378 T7678 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 941378 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 941380 T7724 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21d0f103 name:ZooKeeperConnection Watcher:127.0.0.1:10388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 941380 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 941382 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 941383 T7726 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7962d644 name:ZooKeeperConnection Watcher:127.0.0.1:10388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 941383 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 941387 T7678 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 942390 T7678 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10401_
[junit4] 2> 942391 T7678 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10401_
[junit4] 2> 942401 T7728 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 942401 T7728 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 942402 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 942402 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 942402 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 942402 T7728 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 942404 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 942405 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:10401",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10401_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 942405 T7701 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 942405 T7701 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 942407 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 942408 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 942408 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 942408 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 943403 T7728 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 943404 T7728 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 943405 T7728 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 943405 T7728 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 943406 T7728 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 943406 T7728 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/'
[junit4] 2> 943407 T7728 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 943408 T7728 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 943409 T7728 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 943500 T7728 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 943549 T7728 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 943651 T7728 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 943662 T7728 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 944043 T7728 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 944057 T7728 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 944060 T7728 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 944071 T7728 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 944076 T7728 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 944081 T7728 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 944081 T7728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 944083 T7728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 944083 T7728 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 944084 T7728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 944084 T7728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 944084 T7728 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 944085 T7728 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 944085 T7728 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/data/
[junit4] 2> 944085 T7728 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@185af219
[junit4] 2> 944086 T7728 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/data
[junit4] 2> 944086 T7728 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/data/index/
[junit4] 2> 944087 T7728 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 944087 T7728 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-003/collection1/data/index
[junit4] 2> 944088 T7728 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1809299489, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 944089 T7728 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@78399990 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@569a1815),segFN=segments_1,generation=1}
[junit4] 2> 944089 T7728 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 944095 T7728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 944095 T7728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 944096 T7728 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 944096 T7728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 944096 T7728 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 944096 T7728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 944097 T7728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 944097 T7728 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 944097 T7728 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 944098 T7728 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 944099 T7728 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 944099 T7728 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 944099 T7728 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 944100 T7728 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 944100 T7728 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 944101 T7728 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 944102 T7728 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 944121 T7728 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 944127 T7728 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 944127 T7728 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 944128 T7728 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=31, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 944129 T7728 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@78399990 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@569a1815),segFN=segments_1,generation=1}
[junit4] 2> 944129 T7728 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 944129 T7728 oass.SolrIndexSearcher.<init> Opening Searcher@3b99a7e5[collection1] main
[junit4] 2> 944130 T7728 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 944131 T7728 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 944131 T7728 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 944131 T7728 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 944132 T7728 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 944132 T7728 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 944133 T7728 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 944133 T7728 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 944133 T7728 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 944133 T7728 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 944138 T7729 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b99a7e5[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 944141 T7728 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 944142 T7732 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10401 collection:collection1 shard:shard1
[junit4] 2> 944142 T7678 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 944143 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 944143 T7732 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 944147 T7732 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 944149 T7732 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 944150 T7732 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C426 name=collection1 org.apache.solr.core.SolrCore@38e417c7 url=http://127.0.0.1:10401/collection1 node=127.0.0.1:10401_ C426_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:10401, core=collection1, node_name=127.0.0.1:10401_}
[junit4] 2> 944150 T7732 C426 P10401 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10401/collection1/
[junit4] 2> 944151 T7732 C426 P10401 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 944151 T7732 C426 P10401 oasc.SyncStrategy.syncToMe http://127.0.0.1:10401/collection1/ has no replicas
[junit4] 2> 944151 T7732 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10401/collection1/ shard1
[junit4] 2> 944151 T7732 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 944162 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944163 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 944166 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944166 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944166 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944166 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944169 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 944171 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944273 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944273 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944273 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944310 T7732 oasc.ZkController.register We are http://127.0.0.1:10401/collection1/ and leader is http://127.0.0.1:10401/collection1/
[junit4] 2> 944311 T7732 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10401
[junit4] 2> 944311 T7732 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 944311 T7732 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 944311 T7732 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 944313 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944313 T7732 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 944313 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944313 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944315 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 944315 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:10401",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10401_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 944317 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 944420 T7713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944420 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 944420 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 945588 T7678 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 945589 T7678 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 945592 T7678 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10410
[junit4] 2> 945593 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 945593 T7678 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 945593 T7678 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004
[junit4] 2> 945593 T7678 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/'
[junit4] 2> 945629 T7678 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/solr.xml
[junit4] 2> 945688 T7678 oasc.CoreContainer.<init> New CoreContainer 2011693227
[junit4] 2> 945689 T7678 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/]
[junit4] 2> 945690 T7678 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 945690 T7678 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 945690 T7678 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 945690 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 945691 T7678 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 945691 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 945691 T7678 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 945691 T7678 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 945692 T7678 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 945692 T7678 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 945692 T7678 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 945693 T7678 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 945693 T7678 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 945693 T7678 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:10388/solr
[junit4] 2> 945693 T7678 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 945694 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 945696 T7743 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2084180d name:ZooKeeperConnection Watcher:127.0.0.1:10388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 945696 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 945698 T7678 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 945699 T7745 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bac19b6 name:ZooKeeperConnection Watcher:127.0.0.1:10388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 945699 T7678 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 945703 T7678 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 946706 T7678 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:10410_
[junit4] 2> 946707 T7678 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:10410_
[junit4] 2> 946717 T7747 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 946718 T7747 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 946719 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 946719 T7747 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 946719 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 946719 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 946720 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 946721 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:10410",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10410_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 946721 T7701 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 946721 T7701 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 946723 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 946724 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 946724 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 946724 T7746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 946724 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 947719 T7747 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 947720 T7747 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 947721 T7747 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 947721 T7747 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 947722 T7747 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 947722 T7747 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/'
[junit4] 2> 947724 T7747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 947725 T7747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 947726 T7747 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 947793 T7747 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 947842 T7747 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 947943 T7747 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 947961 T7747 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 948371 T7747 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 948385 T7747 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 948388 T7747 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 948400 T7747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 948405 T7747 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 948410 T7747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 948410 T7747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 948412 T7747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 948412 T7747 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 948413 T7747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 948413 T7747 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 948413 T7747 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 948413 T7747 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 948414 T7747 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/data/
[junit4] 2> 948414 T7747 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@185af219
[junit4] 2> 948415 T7747 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/data
[junit4] 2> 948415 T7747 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/data/index/
[junit4] 2> 948416 T7747 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 948416 T7747 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-004/collection1/data/index
[junit4] 2> 948417 T7747 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1809299489, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 948418 T7747 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@38cf2402 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c21bd8f),segFN=segments_1,generation=1}
[junit4] 2> 948418 T7747 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 948424 T7747 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 948424 T7747 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 948424 T7747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 948425 T7747 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 948425 T7747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 948425 T7747 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 948426 T7747 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 948426 T7747 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 948426 T7747 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 948427 T7747 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 948428 T7747 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 948428 T7747 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 948428 T7747 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 948429 T7747 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 948429 T7747 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 948430 T7747 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 948430 T7747 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 948452 T7747 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 948459 T7747 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 948459 T7747 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 948460 T7747 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=31, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 948460 T7747 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@38cf2402 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c21bd8f),segFN=segments_1,generation=1}
[junit4] 2> 948461 T7747 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 948461 T7747 oass.SolrIndexSearcher.<init> Opening Searcher@7a18f2a[collection1] main
[junit4] 2> 948461 T7747 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 948462 T7747 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 948462 T7747 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 948463 T7747 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 948463 T7747 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 948463 T7747 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 948464 T7747 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 948464 T7747 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 948465 T7747 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 948465 T7747 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 948469 T7748 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a18f2a[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 948472 T7747 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 948473 T7751 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:10410 collection:collection1 shard:shard2
[junit4] 2> 948473 T7678 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 948473 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 948474 T7751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 948478 T7751 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 948479 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948479 T7751 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 948480 T7751 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C427 name=collection1 org.apache.solr.core.SolrCore@2922b606 url=http://127.0.0.1:10410/collection1 node=127.0.0.1:10410_ C427_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:10410, core=collection1, node_name=127.0.0.1:10410_}
[junit4] 2> 948480 T7751 C427 P10410 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:10410/collection1/
[junit4] 2> 948480 T7751 C427 P10410 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 948480 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 948480 T7751 C427 P10410 oasc.SyncStrategy.syncToMe http://127.0.0.1:10410/collection1/ has no replicas
[junit4] 2> 948481 T7751 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:10410/collection1/ shard2
[junit4] 2> 948481 T7751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 948482 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948483 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948483 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948483 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948483 T7746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948486 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 948488 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948590 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948590 T7746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948590 T7712 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948590 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948638 T7751 oasc.ZkController.register We are http://127.0.0.1:10410/collection1/ and leader is http://127.0.0.1:10410/collection1/
[junit4] 2> 948639 T7751 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:10410
[junit4] 2> 948639 T7751 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 948639 T7751 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 948639 T7751 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 948641 T7712 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948641 T7713 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948641 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948641 T7751 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 948642 T7701 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 948643 T7701 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:10410",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:10410_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 948645 T7704 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 948747 T7704 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948747 T7727 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948747 T7711 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 948747 T7746 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 949877 T7678 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 949878 T7678 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 949881 T7678 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:10419
[junit4] 2> 949882 T7678 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 949882 T7678 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 949882 T7678 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-005
[junit4] 2> 949882 T7678 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-005/'
[junit4] 2> 949918 T7678 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-005/solr.xml
[junit4] 2> 949977 T7678 oasc.CoreContainer.<init> New CoreContainer 1703024358
[junit4] 2> 949978 T7678 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/bu
[...truncated too long message...]
hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest3_shard1_replica1/data/index
[junit4] 2> 989966 T7678 oasc.SolrCore.close [ocptest_shardsplit_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@4f8648b2
[junit4] 2> 989974 T7678 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 989975 T7678 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 989975 T7678 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 989975 T7678 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 989976 T7678 oasc.SolrCore.closeSearcher [ocptest_shardsplit_shard2_replica1] Closing main searcher on request.
[junit4] 2> 989976 T7678 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 989977 T7678 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data;done=false>>]
[junit4] 2> 989977 T7678 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data
[junit4] 2> 989977 T7678 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data/index;done=false>>]
[junit4] 2> 989978 T7678 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001/tempDir-006/ocptest_shardsplit_shard2_replica1/data/index
[junit4] 2> 989978 T9208 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:10431_
[junit4] 2> 989980 T7786 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 989981 T9210 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:317)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 989982 T7678 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 994596 T7678 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 994599 T7678 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:10388 10388
[junit4] 2> 994686 T7679 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:10388 10388
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=B2526CA8AF3CA5D1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ko -Dtests.timezone=America/Puerto_Rico -Dtests.file.encoding=UTF-8
[junit4] FAILURE 58.4s J1 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([B2526CA8AF3CA5D1:33B4E2B0D863C5ED]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 994701 T7678 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.MultiThreadedOCPTest-B2526CA8AF3CA5D1-001
[junit4] 2> 58380 T7677 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene49: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ko, timezone=America/Puerto_Rico
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=164516752,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaRuntimeFieldTest, DirectUpdateHandlerOptimizeTest, ZkSolrClientTest, BlockDirectoryTest, TestElisionMultitermQuery, CSVRequestHandlerTest, TestSerializedLuceneMatchVersion, ResourceLoaderTest, NoFacetTest, TestPHPSerializedResponseWriter, TestSolr4Spatial2, TestRealTimeGet, SolrPluginUtilsTest, TestCoreContainer, TestQueryUtils, TestSolrQueryParserResource, TestConfigSets, TestNonDefinedSimilarityFactory, ScriptEngineTest, SolrRequestParserTest, TestCollationField, RequestHandlersTest, TestReversedWildcardFilterFactory, TestDistributedGrouping, TestExtendedDismaxParser, DateRangeFieldTest, SolrTestCaseJ4Test, TestValueSourceCache, AlternateDirectoryTest, HdfsDirectoryTest, TestSuggestSpellingConverter, RemoteQueryErrorTest, StressHdfsTest, AnalysisAfterCoreReloadTest, TestRestManager, TestOrdValues, HttpPartitionTest, TestDistribDocBasedVersion, HdfsChaosMonkeySafeLeaderTest, SortByFunctionTest, TestIndexSearcher, DeleteReplicaTest, TestAtomicUpdateErrorCases, MultiTermTest, TestDocumentBuilder, TestStressLucene, DistributedSuggestComponentTest, SimplePostToolTest, DirectSolrConnectionTest, TestRandomFaceting, SolrInfoMBeanTest, TestLRUCache, TestExpandComponent, HdfsUnloadDistributedZkTest, TermVectorComponentTest, DistanceFunctionTest, TestCharFilters, UUIDUpdateProcessorFallbackTest, TestZkChroot, TestXIncludeConfig, TestLazyCores, SoftAutoCommitTest, TestCollectionAPI, MergeStrategyTest, TestBM25SimilarityFactory, FieldFacetExtrasTest, FileBasedSpellCheckerTest, CollectionsAPIDistributedZkTest, TestImplicitCoreProperties, TestPerFieldSimilarity, CopyFieldTest, OutputWriterTest, SolrCoreTest, TestManagedSchemaFieldResource, MultiThreadedOCPTest]
[junit4] Completed on J1 in 58.66s, 1 test, 1 failure <<< FAILURES!
[...truncated 842 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1754 tests, 1 failure, 53 ignored (19 assumptions)
Total time: 110 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 193858568 bytes
Compression is 0.0%
Took 32 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4785 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4785/
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
No live SolrServers available to handle this request:[https://127.0.0.1:62629]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[https://127.0.0.1:62629]
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:319)
at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:659)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1506)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1527)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.doTest(ChaosMonkeySafeLeaderTest.java:164)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: createcollection the collection time out:180s
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:551)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:340)
at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:301)
... 45 more
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=19891, name=parallelCoreAdminExecutor-4672-thread-5, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19891, name=parallelCoreAdminExecutor-4672-thread-5, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at __randomizedtesting.SeedInfo.seed([2F797331643DA322]:0)
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11498 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/init-core-data-001
[junit4] 2> 316076 T1100 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 316077 T1100 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 316083 T1100 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 316084 T1100 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 316085 T1101 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 316185 T1100 oasc.ZkTestServer.run start zk server on port:54560
[junit4] 2> 316186 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 316190 T1107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39176f51 name:ZooKeeperConnection Watcher:127.0.0.1:54560 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 316190 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 316190 T1100 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 316194 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 316195 T1109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@459d2d23 name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 316195 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 316195 T1100 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 316197 T1100 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 316205 T1100 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 316207 T1100 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 316209 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 316209 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 316212 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 316213 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 316315 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 316316 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 316318 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 316318 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 316320 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 316321 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 316324 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 316324 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 316327 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 316327 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 316330 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 316330 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 316333 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 316333 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 316336 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 316337 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 316339 T1100 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 316340 T1100 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 316344 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 316345 T1111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38ae783 name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 316345 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 317616 T1100 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 317622 T1100 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 317629 T1100 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:54564
[junit4] 2> 317632 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 317632 T1100 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 317632 T1100 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002
[junit4] 2> 317633 T1100 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/'
[junit4] 2> 317663 T1100 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/solr.xml
[junit4] 2> 317720 T1100 oasc.CoreContainer.<init> New CoreContainer 33820033
[junit4] 2> 317721 T1100 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/]
[junit4] 2> 317722 T1100 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 317722 T1100 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 317722 T1100 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 317722 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 317723 T1100 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 317723 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 317723 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 317723 T1100 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 317724 T1100 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 317726 T1100 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 317728 T1100 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 317729 T1100 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 317729 T1100 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 317729 T1100 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54560/solr
[junit4] 2> 317730 T1100 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 317731 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 317732 T1122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7421655e name:ZooKeeperConnection Watcher:127.0.0.1:54560 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 317733 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 317734 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 317735 T1124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28a148cd name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 317736 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 317737 T1100 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 317739 T1100 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 317741 T1100 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 317743 T1100 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 317745 T1100 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 317748 T1100 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 317749 T1100 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54564_
[junit4] 2> 317750 T1100 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54564_
[junit4] 2> 317751 T1100 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 317753 T1100 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 317756 T1100 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:54564_
[junit4] 2> 317756 T1100 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 317757 T1100 oasc.Overseer.start Overseer (id=92193025388249092-127.0.0.1:54564_-n_0000000000) starting
[junit4] 2> 317759 T1100 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 317766 T1126 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 317766 T1100 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 317768 T1100 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 317769 T1100 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 317771 T1125 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 317775 T1127 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 317775 T1127 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 317776 T1127 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 317776 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 317777 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 317778 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54564",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54564_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 317779 T1125 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 317779 T1125 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 317781 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 317781 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 318776 T1127 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 318777 T1127 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 318778 T1127 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 318778 T1127 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 318779 T1127 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 318779 T1127 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/collection1/'
[junit4] 2> 318781 T1127 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 318782 T1127 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 318782 T1127 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 318851 T1127 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 318916 T1127 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 319018 T1127 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 319030 T1127 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 319402 T1127 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 319405 T1127 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 319407 T1127 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 319412 T1127 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 319434 T1127 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 319434 T1127 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 319503 T1127 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-002/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/
[junit4] 2> 319504 T1127 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f94dadb
[junit4] 2> 319506 T1127 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data
[junit4] 2> 319507 T1127 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/index/
[junit4] 2> 319508 T1127 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 319510 T1127 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/index
[junit4] 2> 319511 T1127 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=74.529296875, floorSegmentMB=0.826171875, forceMergeDeletesPctAllowed=22.404707175441295, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 319517 T1127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 319517 T1127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 319523 T1127 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 319524 T1127 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 319524 T1127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 319524 T1127 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 319525 T1127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 319525 T1127 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 319525 T1127 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 319526 T1127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 319526 T1127 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 319527 T1127 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 319527 T1127 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 319527 T1127 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 319528 T1127 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 319528 T1127 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 319529 T1127 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 319529 T1127 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 319530 T1127 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 319548 T1127 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 319552 T1127 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 319552 T1127 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 319553 T1127 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=53.6953125, floorSegmentMB=1.2109375, forceMergeDeletesPctAllowed=6.5127932328050235, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 319555 T1127 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 319556 T1127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 319556 T1127 oass.SolrIndexSearcher.<init> Opening Searcher@5a8126d1[collection1] main
[junit4] 2> 319556 T1127 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 319557 T1127 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 319558 T1127 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 319558 T1127 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 319558 T1127 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 319559 T1127 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 319559 T1127 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 319560 T1127 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 319560 T1127 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 319562 T1127 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 319563 T1127 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 319563 T1127 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 319566 T1129 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a8126d1[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 319569 T1127 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 319569 T1132 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54564 collection:control_collection shard:shard1
[junit4] 2> 319570 T1100 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 319570 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 319571 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 319575 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 319575 T1132 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 319577 T1134 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3264fc02 name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 319577 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 319578 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319578 T1100 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 319579 T1132 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 319579 T1132 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C169 name=collection1 org.apache.solr.core.SolrCore@205b860e url=https://127.0.0.1:54564/collection1 node=127.0.0.1:54564_ C169_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:54564, core=collection1, node_name=127.0.0.1:54564_}
[junit4] 2> 319579 T1132 C169 P54564 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54564/collection1/
[junit4] 2> 319579 T1132 C169 P54564 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 319580 T1132 C169 P54564 oasc.SyncStrategy.syncToMe https://127.0.0.1:54564/collection1/ has no replicas
[junit4] 2> 319580 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 319580 T1132 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54564/collection1/ shard1
[junit4] 2> 319580 T1132 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 319581 T1100 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 319583 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319583 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 319584 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 319587 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 319590 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319692 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 319692 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 319739 T1132 oasc.ZkController.register We are https://127.0.0.1:54564/collection1/ and leader is https://127.0.0.1:54564/collection1/
[junit4] 2> 319740 T1132 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54564
[junit4] 2> 319740 T1132 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 319740 T1132 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 319740 T1132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 319742 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319742 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319742 T1132 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 319742 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319744 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 319745 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:54564",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54564_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 319747 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 319849 T1137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 319849 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 320504 T1100 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 320505 T1100 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 320511 T1100 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 320513 T1100 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:54568
[junit4] 2> 320515 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 320515 T1100 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 320516 T1100 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003
[junit4] 2> 320516 T1100 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/'
[junit4] 2> 320547 T1100 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/solr.xml
[junit4] 2> 320603 T1100 oasc.CoreContainer.<init> New CoreContainer 1220373088
[junit4] 2> 320603 T1100 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/]
[junit4] 2> 320604 T1100 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 320604 T1100 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 320605 T1100 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 320605 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 320605 T1100 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 320605 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 320606 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 320606 T1100 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 320606 T1100 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 320609 T1100 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 320611 T1100 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 320611 T1100 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 320611 T1100 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 320612 T1100 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54560/solr
[junit4] 2> 320612 T1100 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 320613 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 320614 T1148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a5b860e name:ZooKeeperConnection Watcher:127.0.0.1:54560 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 320616 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 320618 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 320619 T1150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d85af06 name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 320619 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 320626 T1100 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 321629 T1100 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54568_
[junit4] 2> 321630 T1100 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54568_
[junit4] 2> 321642 T1152 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 321643 T1152 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 321644 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 321644 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 321644 T1152 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 321644 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 321645 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 321646 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:54568",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54568_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 321646 T1125 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 321646 T1125 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 321649 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 321649 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 321649 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 321649 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 322644 T1152 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 322645 T1152 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 322646 T1152 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 322646 T1152 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 322647 T1152 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 322647 T1152 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/collection1/'
[junit4] 2> 322649 T1152 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 322650 T1152 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 322651 T1152 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 322717 T1152 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 322766 T1152 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 322868 T1152 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 322880 T1152 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 323260 T1152 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 323264 T1152 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 323266 T1152 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 323271 T1152 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 323295 T1152 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 323296 T1152 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 323296 T1152 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-003/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/
[junit4] 2> 323296 T1152 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f94dadb
[junit4] 2> 323298 T1152 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1
[junit4] 2> 323299 T1152 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/index/
[junit4] 2> 323300 T1152 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 323302 T1152 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/index
[junit4] 2> 323303 T1152 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=74.529296875, floorSegmentMB=0.826171875, forceMergeDeletesPctAllowed=22.404707175441295, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 323309 T1152 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 323310 T1152 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 323317 T1152 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 323318 T1152 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 323318 T1152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 323318 T1152 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 323319 T1152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 323319 T1152 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 323319 T1152 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 323320 T1152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 323320 T1152 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 323321 T1152 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 323321 T1152 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 323321 T1152 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 323322 T1152 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 323323 T1152 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 323323 T1152 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 323324 T1152 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 323324 T1152 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 323344 T1152 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 323348 T1152 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 323348 T1152 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 323349 T1152 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=53.6953125, floorSegmentMB=1.2109375, forceMergeDeletesPctAllowed=6.5127932328050235, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 323352 T1152 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 323352 T1152 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 323352 T1152 oass.SolrIndexSearcher.<init> Opening Searcher@3c025678[collection1] main
[junit4] 2> 323353 T1152 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 323354 T1152 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 323354 T1152 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 323355 T1152 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 323355 T1152 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 323355 T1152 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 323356 T1152 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 323357 T1152 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 323357 T1152 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 323357 T1152 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 323362 T1153 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3c025678[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 323365 T1152 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 323366 T1156 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:54568 collection:collection1 shard:shard1
[junit4] 2> 323367 T1100 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 323367 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 323367 T1156 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 323374 T1156 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 323375 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323376 T1156 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 323376 T1156 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C170 name=collection1 org.apache.solr.core.SolrCore@5b2ae2a8 url=https://127.0.0.1:54568/collection1 node=127.0.0.1:54568_ C170_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:54568, core=collection1, node_name=127.0.0.1:54568_}
[junit4] 2> 323376 T1156 C170 P54568 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:54568/collection1/
[junit4] 2> 323377 T1156 C170 P54568 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 323377 T1156 C170 P54568 oasc.SyncStrategy.syncToMe https://127.0.0.1:54568/collection1/ has no replicas
[junit4] 2> 323377 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 323377 T1156 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:54568/collection1/ shard1
[junit4] 2> 323378 T1156 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 323380 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323381 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323381 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323381 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323385 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 323387 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323490 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323490 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323490 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323536 T1156 oasc.ZkController.register We are https://127.0.0.1:54568/collection1/ and leader is https://127.0.0.1:54568/collection1/
[junit4] 2> 323537 T1156 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:54568
[junit4] 2> 323537 T1156 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 323537 T1156 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 323537 T1156 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 323539 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323539 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323539 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323539 T1156 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 323540 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 323541 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:54568",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:54568_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 323543 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 323683 T1136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323683 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 323683 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 324622 T1100 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 324623 T1100 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 324628 T1100 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 324631 T1100 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:21658
[junit4] 2> 324633 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 324634 T1100 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 324634 T1100 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004
[junit4] 2> 324634 T1100 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/'
[junit4] 2> 324664 T1100 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/solr.xml
[junit4] 2> 324717 T1100 oasc.CoreContainer.<init> New CoreContainer 1938728225
[junit4] 2> 324718 T1100 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/]
[junit4] 2> 324719 T1100 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 324719 T1100 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 324719 T1100 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 324719 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 324720 T1100 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 324720 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 324720 T1100 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 324720 T1100 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 324721 T1100 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 324723 T1100 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 324725 T1100 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 324726 T1100 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 324726 T1100 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 324726 T1100 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54560/solr
[junit4] 2> 324726 T1100 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 324727 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 324729 T1167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@435c1bc2 name:ZooKeeperConnection Watcher:127.0.0.1:54560 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 324730 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 324732 T1100 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 324733 T1169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71c613c3 name:ZooKeeperConnection Watcher:127.0.0.1:54560/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 324733 T1100 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 324739 T1100 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 325743 T1100 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21658_
[junit4] 2> 325744 T1100 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21658_
[junit4] 2> 325756 T1171 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 325757 T1171 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 325758 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 325758 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 325758 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 325758 T1171 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 325759 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 325760 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:21658",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:21658_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 325760 T1125 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 325761 T1125 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 325763 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 325763 T1137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 325764 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 325764 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 325763 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 326759 T1171 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 326760 T1171 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 326761 T1171 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 326761 T1171 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 326762 T1171 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 326762 T1171 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/collection1/'
[junit4] 2> 326764 T1171 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 326765 T1171 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 326766 T1171 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 326833 T1171 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
[junit4] 2> 326882 T1171 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 326983 T1171 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 326994 T1171 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 327352 T1171 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 327356 T1171 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 327357 T1171 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 327363 T1171 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 327404 T1171 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 327405 T1171 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 327405 T1171 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-004/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/
[junit4] 2> 327405 T1171 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f94dadb
[junit4] 2> 327407 T1171 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2
[junit4] 2> 327408 T1171 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/index/
[junit4] 2> 327409 T1171 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 327411 T1171 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/index
[junit4] 2> 327412 T1171 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=74.529296875, floorSegmentMB=0.826171875, forceMergeDeletesPctAllowed=22.404707175441295, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 327418 T1171 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 327418 T1171 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 327425 T1171 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 327425 T1171 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 327426 T1171 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 327426 T1171 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 327426 T1171 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 327427 T1171 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 327427 T1171 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 327427 T1171 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 327428 T1171 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 327428 T1171 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 327429 T1171 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 327429 T1171 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 327430 T1171 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 327430 T1171 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 327431 T1171 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 327431 T1171 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 327432 T1171 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 327453 T1171 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 327456 T1171 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 327457 T1171 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 327458 T1171 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=53.6953125, floorSegmentMB=1.2109375, forceMergeDeletesPctAllowed=6.5127932328050235, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 327460 T1171 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 327460 T1171 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 327461 T1171 oass.SolrIndexSearcher.<init> Opening Searcher@5079d963[collection1] main
[junit4] 2> 327461 T1171 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 327462 T1171 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 327463 T1171 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 327463 T1171 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 327463 T1171 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 327464 T1171 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 327465 T1171 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 327465 T1171 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 327465 T1171 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 327466 T1171 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 327470 T1172 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5079d963[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 327475 T1171 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 327476 T1175 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:21658 collection:collection1 shard:shard2
[junit4] 2> 327476 T1100 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1
[junit4] 2> 327477 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 327477 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 327483 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 327485 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327485 T1175 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 327485 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C171 name=collection1 org.apache.solr.core.SolrCore@5cc91e40 url=https://127.0.0.1:21658/collection1 node=127.0.0.1:21658_ C171_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:21658, core=collection1, node_name=127.0.0.1:21658_}
[junit4] 2> 327485 T1175 C171 P21658 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:21658/collection1/
[junit4] 2> 327486 T1175 C171 P21658 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 327486 T1175 C171 P21658 oasc.SyncStrategy.syncToMe https://127.0.0.1:21658/collection1/ has no replicas
[junit4] 2> 327486 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 327486 T1175 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:21658/collection1/ shard2
[junit4] 2> 327487 T1175 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 327489 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327490 T1137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327490 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327490 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327490 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327493 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 327496 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327598 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327598 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327598 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327598 T1137 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327645 T1175 oasc.ZkController.register We are https://127.0.0.1:21658/collection1/ and leader is https://127.0.0.1:21658/collection1/
[junit4] 2> 327646 T1175 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:21658
[junit4] 2> 327646 T1175 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 327646 T1175 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 327646 T1175 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 327648 T1137 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327648 T1136 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327648 T1175 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 327648 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327649 T1125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 327650 T1125 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:21658",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:21658_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 327652 T1128 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 327755 T1135 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327755 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327755 T1170 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 327755 T1151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 328889 T1100 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 328890 T1100 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 328895 T1100 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 328898 T1100 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:63174
[junit4] 2> 328900 T1100 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 328901 T1100 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 328901 T1100 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-005
[junit4] 2> 328901 T1100 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-005/'
[junit4] 2> 328931 T1100 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J1/./temp/solr.cloud.ChaosMonkeySafeLeaderTest-2F797331643DA322-001/tempDir-005/solr.xml
[junit4] 2> 328987 T1100 oasc.CoreC
[...truncated too long message...]
seer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:289)
[junit4] 2>
[junit4] 2> 4107774 T18385 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 4111943 T18385 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 4111946 T18385 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37764 37764
[junit4] 2> 4112044 T18386 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:37764 37764
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=2F797331643DA322 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=zh_CN -Dtests.timezone=America/Iqaluit -Dtests.file.encoding=US-ASCII
[junit4] ERROR 62.2s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19891, name=parallelCoreAdminExecutor-4672-thread-5, state=RUNNABLE, group=TGRP-MultiThreadedOCPTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([2F797331643DA322]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4112066 T18385 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build/solr-core/test/J0/./temp/solr.cloud.MultiThreadedOCPTest-2F797331643DA322-001
[junit4] 2> 62196 T18384 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene49: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=zh_CN, timezone=America/Iqaluit
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=237237968,total=535822336
[junit4] 2> NOTE: All tests run in this JVM: [TestAnalyzeInfixSuggestions, IndexSchemaTest, ShardRoutingTest, DistributedSpellCheckComponentTest, FileUtilsTest, TestDocBasedVersionConstraints, TestStressReorder, AddSchemaFieldsUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, RequiredFieldsTest, TestDFRSimilarityFactory, LeaderElectionTest, TestSearchPerf, FieldFacetTest, FunctionTest, TestDocSet, BadComponentTest, TestSolrDeletionPolicy1, TestSearcherReuse, TestRandomDVFaceting, ResponseHeaderTest, ParsingFieldUpdateProcessorsTest, OverseerTest, TestReloadAndDeleteDocs, DistribDocExpirationUpdateProcessorTest, DirectSolrSpellCheckerTest, TestArbitraryIndexDir, JsonLoaderTest, NoCacheHeaderTest, TestCollationFieldDocValues, TestExceedMaxTermLength, TestQuerySenderListener, PluginInfoTest, IndexBasedSpellCheckerTest, TestManagedSynonymFilterFactory, TestDistributedSearch, HighlighterConfigTest, TestDynamicFieldCollectionResource, TestHashPartitioner, DirectUpdateHandlerTest, TestManagedStopFilterFactory, TestSort, OpenExchangeRatesOrgProviderTest, TriLevelCompositeIdRoutingTest, HighlighterTest, AbstractAnalyticsStatsTest, TestFastWriter, TestRequestStatusCollectionAPI, SyncSliceTest, SuggesterTest, WordBreakSolrSpellCheckerTest, TestReload, TestSolr4Spatial, OverseerStatusTest, TestMissingGroups, XsltUpdateRequestHandlerTest, TestBinaryResponseWriter, SignatureUpdateProcessorFactoryTest, PathHierarchyTokenizerFactoryTest, TestSolrJ, TestCustomSort, TestReplicationHandler, TestLMDirichletSimilarityFactory, ExpressionTest, TestWordDelimiterFilterFactory, SimpleFacetsTest, SolrXmlInZkTest, TestStressVersions, TestLMJelinekMercerSimilarityFactory, HighlighterMaxOffsetTest, NotRequiredUniqueKeyTest, SSLMigrationTest, SuggesterWFSTTest, BasicFunctionalityTest, CoreAdminHandlerTest, SpellPossibilityIteratorTest, TermsComponentTest, BasicDistributedZkTest, TestStandardQParsers, TestFastOutputStream, CoreAdminCreateDiscoverTest, MoreLikeThisHandlerTest, ChaosMonkeyNothingIsSafeTest, TestManagedResource, FastVectorHighlighterTest, TestFastLRUCache, TestDefaultSimilarityFactory, TestSolrCoreProperties, EchoParamsTest, SolrCoreCheckLockOnStartupTest, HdfsBasicDistributedZk2Test, BadIndexSchemaTest, TestSolrXml, TestSchemaResource, ZkControllerTest, JSONWriterTest, XmlUpdateRequestHandlerTest, TestComplexPhraseQParserPlugin, ReplicationFactorTest, TestCoreDiscovery, TestSolrDeletionPolicy2, BasicDistributedZk2Test, UnloadDistributedZkTest, OpenCloseCoreStressTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, TestZkChroot, ZkCLITest, AutoCommitTest, TestFaceting, TestJoin, TestMultiCoreConfBootstrap, HardAutoCommitTest, TestRangeQuery, SpellCheckComponentTest, SolrCmdDistributorTest, PeerSyncTest, ConvertedLegacyTest, OverseerCollectionProcessorTest, TestIndexSearcher, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, SpellCheckCollatorTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, TestTrie, SpatialFilterTest, TestCSVLoader, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestQueryTypes, PrimitiveFieldTypeTest, TestOmitPositions, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, TestJmxIntegration, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, MBeansHandlerTest, QueryParsingTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, PingRequestHandlerTest, CSVRequestHandlerTest, TestComponentsName, TestLFUCache, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, ResponseLogComponentTest, TestStressRecovery, MultiTermTest, SampleTest, TestBinaryField, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCodecSupport, SynonymTokenizerTest, TestPerFieldSimilarity, TimeZoneUtilsTest, URLClassifyProcessorTest, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, DateFieldTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, ClusterStateTest, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, CircularListTest, CursorPagingTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, AbstractAnalyticsFacetTest, QueryFacetTest, AssignTest, AsyncMigrateRouteKeyTest, CollectionsAPIAsyncDistributedZkTest, CustomCollectionTest, DeleteInactiveReplicaTest, DeleteReplicaTest, MigrateRouteKeyTest, MultiThreadedOCPTest]
[junit4] Completed on J0 in 63.23s, 1 test, 1 error <<< FAILURES!
[...truncated 331 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1298: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:922: There were test failures: 422 suites, 1754 tests, 2 errors, 53 ignored (19 assumptions)
Total time: 125 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-trunk-Java7 #4781
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 200598785 bytes
Compression is 0.0%
Took 33 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure