You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/08/01 00:21:04 UTC
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 588 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/588/
3 tests failed.
REGRESSION: org.apache.solr.handler.admin.InfoHandlerTest.testOverriddenHandlers
Error Message:
posix_spawn is not a supported process launch mechanism on this platform.
Stack Trace:
java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
at __randomizedtesting.SeedInfo.seed([C27FC29631BDB979:2424CEB1C8F476CD]:0)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
at java.security.AccessController.doPrivileged(Native Method)
at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
at java.lang.Runtime.exec(Runtime.java:617)
at java.lang.Runtime.exec(Runtime.java:450)
at java.lang.Runtime.exec(Runtime.java:347)
at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
at org.apache.solr.handler.admin.InfoHandlerTest$CountSystemInfoHandler.handleRequestBody(InfoHandlerTest.java:157)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
at org.apache.solr.handler.admin.InfoHandlerTest.handleRequest(InfoHandlerTest.java:168)
at org.apache.solr.handler.admin.InfoHandlerTest.testOverriddenHandlers(InfoHandlerTest.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
REGRESSION: org.apache.solr.request.TestIntervalFaceting.testIntFieldsMultipleSegments
Error Message:
Expected multiple reader leaves. Found 1
Stack Trace:
java.lang.AssertionError: Expected multiple reader leaves. Found 1
at __randomizedtesting.SeedInfo.seed([C27FC29631BDB979:C7DC944975E7E1A0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.request.TestIntervalFaceting.assertMultipleReaders(TestIntervalFaceting.java:134)
at org.apache.solr.request.TestIntervalFaceting.testIntFieldsMultipleSegments(TestIntervalFaceting.java:711)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([C27FC29631BDB979:43994C8E46E2D945]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11739 lines...]
[junit4] Suite: org.apache.solr.handler.admin.InfoHandlerTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001
[junit4] 2> 317086 T15213 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 317088 T15213 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 317088 T15213 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 317090 T15213 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 317090 T15213 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 317091 T15213 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 317227 T15213 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 317296 T15213 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 317297 T15213 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 317310 T15213 oass.IndexSchema.readSchema [null] Schema name=test
[junit4] 2> 317666 T15213 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 317667 T15213 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 317668 T15213 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 317677 T15213 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 317679 T15213 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 317687 T15213 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 317691 T15213 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 317693 T15213 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 317694 T15213 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 317694 T15213 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 317695 T15213 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 317695 T15213 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 317695 T15213 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 317695 T15213 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 317696 T15213 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr
[junit4] 2> 317696 T15213 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/'
[junit4] 2> 317734 T15213 oasc.CoreContainer.<init> New CoreContainer 917843536
[junit4] 2> 317734 T15213 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/]
[junit4] 2> 317735 T15213 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 317735 T15213 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 317735 T15213 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 317736 T15213 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 317739 T15213 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=false
[junit4] 2> 317740 T15213 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 317740 T15213 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 317740 T15213 oasc.CoreContainer.load Host Name:
[junit4] 2> 317745 T15214 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 317746 T15214 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 317746 T15214 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 317747 T15214 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 317804 T15214 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 317866 T15214 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 317867 T15214 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 317879 T15214 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 318266 T15214 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 318266 T15214 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 318267 T15214 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 318275 T15214 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 318278 T15214 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 318289 T15214 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 318293 T15214 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 318297 T15214 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 318297 T15214 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 318298 T15214 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 318298 T15214 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 318298 T15214 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 318299 T15214 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 318299 T15214 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/
[junit4] 2> 318299 T15214 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 318300 T15214 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/
[junit4] 2> 318300 T15214 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13bab1a1
[junit4] 2> 318301 T15214 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001
[junit4] 2> 318301 T15214 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index/
[junit4] 2> 318301 T15214 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index' doesn't exist. Creating new index...
[junit4] 2> 318302 T15214 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index
[junit4] 2> 318302 T15214 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 318303 T15214 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(BaseDirectoryWrapper(RAMDirectory@4ca5b545 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c7e47da)),segFN=segments_1,generation=1}
[junit4] 2> 318303 T15214 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 318304 T15214 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 318309 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 318309 T15214 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 318309 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 318310 T15214 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 318310 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 318310 T15214 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 318310 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 318311 T15214 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
[junit4] 2> 318311 T15214 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
[junit4] 2> 318311 T15214 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 318311 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 318312 T15214 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 318312 T15214 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 318313 T15214 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 318313 T15214 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 318313 T15214 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 318314 T15214 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 318314 T15214 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 318315 T15214 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 318315 T15214 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 318316 T15214 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 318316 T15214 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 318317 T15214 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 318317 T15214 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 318317 T15214 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 318318 T15214 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 318318 T15214 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318319 T15214 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318319 T15214 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318320 T15214 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318320 T15214 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318321 T15214 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318321 T15214 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318322 T15214 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 318322 T15214 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 318323 T15214 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 318341 T15214 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 318343 T15214 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 318345 T15214 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 318346 T15214 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 318346 T15214 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=4.408203125, floorSegmentMB=1.296875, forceMergeDeletesPctAllowed=11.098524312956638, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 318347 T15214 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(BaseDirectoryWrapper(RAMDirectory@4ca5b545 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7c7e47da)),segFN=segments_1,generation=1}
[junit4] 2> 318347 T15214 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 318348 T15214 oass.SolrIndexSearcher.<init> Opening Searcher@407756ef[collection1] main
[junit4] 2> 318348 T15214 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 318348 T15214 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 318349 T15214 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 318349 T15214 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
[junit4] 2> 318349 T15214 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 318349 T15214 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 318358 T15214 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 318465 T15215 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@407756ef[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 318469 T15214 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 318471 T15213 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 318475 T15213 oas.SolrTestCaseJ4.setUp ###Starting testCoreAdminHandler
[junit4] 2> 318488 T15213 oas.SolrTestCaseJ4.tearDown ###Ending testCoreAdminHandler
[junit4] 2> 318492 T15213 oas.SolrTestCaseJ4.setUp ###Starting testOverriddenHandlers
[junit4] 2> 318502 T15213 oas.SolrTestCaseJ4.tearDown ###Ending testOverriddenHandlers
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=InfoHandlerTest -Dtests.method=testOverriddenHandlers -Dtests.seed=C27FC29631BDB979 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=tr -Dtests.timezone=CST -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.02s J1 | InfoHandlerTest.testOverriddenHandlers <<<
[junit4] > Throwable #1: java.lang.Error: posix_spawn is not a supported process launch mechanism on this platform.
[junit4] > at __randomizedtesting.SeedInfo.seed([C27FC29631BDB979:2424CEB1C8F476CD]:0)
[junit4] > at java.lang.UNIXProcess$1.run(UNIXProcess.java:111)
[junit4] > at java.lang.UNIXProcess$1.run(UNIXProcess.java:93)
[junit4] > at java.security.AccessController.doPrivileged(Native Method)
[junit4] > at java.lang.UNIXProcess.<clinit>(UNIXProcess.java:91)
[junit4] > at java.lang.ProcessImpl.start(ProcessImpl.java:130)
[junit4] > at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
[junit4] > at java.lang.Runtime.exec(Runtime.java:617)
[junit4] > at java.lang.Runtime.exec(Runtime.java:450)
[junit4] > at java.lang.Runtime.exec(Runtime.java:347)
[junit4] > at org.apache.solr.handler.admin.SystemInfoHandler.execute(SystemInfoHandler.java:220)
[junit4] > at org.apache.solr.handler.admin.SystemInfoHandler.getSystemInfo(SystemInfoHandler.java:176)
[junit4] > at org.apache.solr.handler.admin.SystemInfoHandler.handleRequestBody(SystemInfoHandler.java:97)
[junit4] > at org.apache.solr.handler.admin.InfoHandlerTest$CountSystemInfoHandler.handleRequestBody(InfoHandlerTest.java:157)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
[junit4] > at org.apache.solr.handler.admin.InfoHandler.handleRequestBody(InfoHandler.java:88)
[junit4] > at org.apache.solr.handler.admin.InfoHandlerTest.handleRequest(InfoHandlerTest.java:168)
[junit4] > at org.apache.solr.handler.admin.InfoHandlerTest.testOverriddenHandlers(InfoHandlerTest.java:98)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 318511 T15213 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 318511 T15213 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=917843536
[junit4] 2> 318511 T15213 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@202e35b5
[junit4] 2> 318523 T15213 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 318524 T15213 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 318524 T15213 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 318524 T15213 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 318526 T15213 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 318527 T15213 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 318527 T15213 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index;done=false>>]
[junit4] 2> 318527 T15213 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001/index
[junit4] 2> 318527 T15213 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001;done=false>>]
[junit4] 2> 318528 T15213 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001/init-core-data-001
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.handler.admin.InfoHandlerTest-C27FC29631BDB979-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=tr, timezone=CST
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=81025280,total=509607936
[junit4] 2> NOTE: All tests run in this JVM: [SolrRequestParserTest, TestQuerySenderNoQuery, TestAtomicUpdateErrorCases, TestSolrJ, TestManagedResourceStorage, BJQParserTest, TestUniqueKeyFieldResource, ZkSolrClientTest, TestCollectionAPI, TestRecoveryHdfs, DistribCursorPagingTest, TestFreeTextSuggestions, SuggestComponentTest, TestOverriddenPrefixQueryForCustomFieldType, CoreAdminRequestStatusTest, TestUpdate, AnalyticsMergeStrategyTest, BlockDirectoryTest, TestDocBasedVersionConstraints, InfoHandlerTest]
[junit4] Completed on J1 in 1.47s, 2 tests, 1 error <<< FAILURES!
[...truncated 57 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/init-core-data-001
[junit4] 2> 480716 T13341 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 480716 T13341 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /twi/
[junit4] 2> 480723 T13341 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 480724 T13341 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 480725 T13342 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 480825 T13341 oasc.ZkTestServer.run start zk server on port:58856
[junit4] 2> 480826 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 480831 T13348 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56f84791 name:ZooKeeperConnection Watcher:127.0.0.1:58856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 480831 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 480832 T13341 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 480836 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 480837 T13350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@53045f59 name:ZooKeeperConnection Watcher:127.0.0.1:58856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 480837 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 480838 T13341 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 480841 T13341 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 480843 T13341 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 480845 T13341 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 480847 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 480848 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 480852 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 480853 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 480955 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 480956 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 480958 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 480959 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 480962 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 480962 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 480965 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 480965 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 480968 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 480968 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 480971 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 480971 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 480974 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 480975 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 480977 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 480978 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 480980 T13341 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 480981 T13341 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 480985 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 480987 T13352 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bfee644 name:ZooKeeperConnection Watcher:127.0.0.1:58856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 480987 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 482050 T13341 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 482054 T13341 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 482057 T13341 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:50649
[junit4] 2> 482059 T13341 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 482059 T13341 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 482059 T13341 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002
[junit4] 2> 482060 T13341 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/'
[junit4] 2> 482088 T13341 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/solr.xml
[junit4] 2> 482140 T13341 oasc.CoreContainer.<init> New CoreContainer 534339597
[junit4] 2> 482141 T13341 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/]
[junit4] 2> 482141 T13341 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 482142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 482142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 482142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 482142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 482142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 482143 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 482143 T13341 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 482143 T13341 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 482145 T13341 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 482147 T13341 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 482147 T13341 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 482147 T13341 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 482147 T13341 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58856/solr
[junit4] 2> 482148 T13341 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 482148 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 482150 T13363 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30e55dcb name:ZooKeeperConnection Watcher:127.0.0.1:58856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 482150 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 482153 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 482154 T13365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5bdd7202 name:ZooKeeperConnection Watcher:127.0.0.1:58856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 482154 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 482156 T13341 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 482159 T13341 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 482161 T13341 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 482164 T13341 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 482166 T13341 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 482169 T13341 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 482171 T13341 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50649_twi
[junit4] 2> 482172 T13341 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50649_twi
[junit4] 2> 482174 T13341 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 482176 T13341 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 482179 T13341 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50649_twi
[junit4] 2> 482179 T13341 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 482181 T13341 oasc.Overseer.start Overseer (id=92198614292103172-127.0.0.1:50649_twi-n_0000000000) starting
[junit4] 2> 482183 T13341 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 482191 T13367 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 482192 T13341 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 482194 T13341 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 482195 T13341 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 482198 T13366 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 482202 T13368 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 482202 T13368 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 482203 T13368 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 482203 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 482204 T13366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 482205 T13366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:50649/twi",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50649_twi",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 482205 T13366 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 482205 T13366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 482207 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 482208 T13369 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 483204 T13368 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 483205 T13368 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 483206 T13368 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 483206 T13368 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 483207 T13368 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 483207 T13368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/collection1/'
[junit4] 2> 483209 T13368 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 483209 T13368 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 483210 T13368 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 483274 T13368 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 483334 T13368 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 483436 T13368 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 483451 T13368 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 483883 T13368 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 483884 T13368 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 483885 T13368 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 483899 T13368 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 483902 T13368 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 483918 T13368 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 483923 T13368 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 483927 T13368 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 483928 T13368 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 483928 T13368 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 483928 T13368 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 483929 T13368 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 483929 T13368 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 483929 T13368 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 483930 T13368 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 483930 T13368 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-002/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-001/control/data/
[junit4] 2> 483930 T13368 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@434de008
[junit4] 2> 483931 T13368 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-001/control/data
[junit4] 2> 483931 T13368 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-001/control/data/index/
[junit4] 2> 483932 T13368 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 483932 T13368 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-001/control/data/index
[junit4] 2> 483932 T13368 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=32, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6061597615635959]
[junit4] 2> 483933 T13368 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ab89df5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ef24f2c),segFN=segments_1,generation=1}
[junit4] 2> 483933 T13368 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 483940 T13368 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 483940 T13368 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 483941 T13368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 483941 T13368 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 483941 T13368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 483941 T13368 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 483942 T13368 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 483942 T13368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 483943 T13368 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 483943 T13368 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 483944 T13368 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 483944 T13368 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 483944 T13368 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 483945 T13368 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 483945 T13368 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 483946 T13368 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 483965 T13368 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 483970 T13368 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 483970 T13368 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 483971 T13368 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=34.5966796875, floorSegmentMB=2.1396484375, forceMergeDeletesPctAllowed=29.483226279920693, segmentsPerTier=9.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 483972 T13368 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1ab89df5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@ef24f2c),segFN=segments_1,generation=1}
[junit4] 2> 483972 T13368 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 483973 T13368 oass.SolrIndexSearcher.<init> Opening Searcher@739ee290[collection1] main
[junit4] 2> 483973 T13368 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 483974 T13368 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 483974 T13368 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 483975 T13368 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 483975 T13368 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 483975 T13368 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 483976 T13368 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 483976 T13368 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 483977 T13368 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 483979 T13368 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 483979 T13368 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 483979 T13368 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 483983 T13370 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@739ee290[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 483987 T13368 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 483987 T13373 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:50649/twi collection:control_collection shard:shard1
[junit4] 2> 483989 T13341 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 483989 T13341 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 483989 T13373 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 483994 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 483995 T13373 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 483996 T13375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bb77724 name:ZooKeeperConnection Watcher:127.0.0.1:58856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 483996 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 483997 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 483997 T13341 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 483998 T13373 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 483998 T13373 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C332 name=collection1 org.apache.solr.core.SolrCore@3bbe3000 url=https://127.0.0.1:50649/twi/collection1 node=127.0.0.1:50649_twi C332_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:50649/twi, core=collection1, node_name=127.0.0.1:50649_twi}
[junit4] 2> 483998 T13373 C332 P50649 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:50649/twi/collection1/
[junit4] 2> 483998 T13373 C332 P50649 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 483999 T13366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 483999 T13373 C332 P50649 oasc.SyncStrategy.syncToMe https://127.0.0.1:50649/twi/collection1/ has no replicas
[junit4] 2> 483999 T13373 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:50649/twi/collection1/ shard1
[junit4] 2> 483999 T13373 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 484000 T13341 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 484002 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484003 T13369 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 484003 T13376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 484007 T13366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 484009 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484113 T13369 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 484113 T13376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 484161 T13373 oasc.ZkController.register We are https://127.0.0.1:50649/twi/collection1/ and leader is https://127.0.0.1:50649/twi/collection1/
[junit4] 2> 484162 T13373 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50649/twi
[junit4] 2> 484162 T13373 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 484162 T13373 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 484162 T13373 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 484164 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484164 T13377 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484165 T13373 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 484164 T13378 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484166 T13366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 484167 T13366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:50649/twi",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:50649_twi",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 484169 T13378 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 484272 T13378 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 484272 T13376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 485051 T13341 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 485053 T13341 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 485056 T13341 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 485057 T13341 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:45041
[junit4] 2> 485059 T13341 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 485060 T13341 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 485060 T13341 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003
[junit4] 2> 485060 T13341 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/'
[junit4] 2> 485089 T13341 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/solr.xml
[junit4] 2> 485139 T13341 oasc.CoreContainer.<init> New CoreContainer 951945456
[junit4] 2> 485140 T13341 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/]
[junit4] 2> 485141 T13341 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 485141 T13341 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 485141 T13341 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 485141 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 485142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 485142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 485142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 485142 T13341 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 485143 T13341 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 485144 T13341 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 485146 T13341 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 485146 T13341 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 485146 T13341 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 485147 T13341 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58856/solr
[junit4] 2> 485147 T13341 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 485148 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 485149 T13389 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@614dd486 name:ZooKeeperConnection Watcher:127.0.0.1:58856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 485150 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 485152 T13341 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 485153 T13391 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@310ad9ed name:ZooKeeperConnection Watcher:127.0.0.1:58856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 485153 T13341 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 485158 T13341 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 486162 T13341 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45041_twi
[junit4] 2> 486163 T13341 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45041_twi
[junit4] 2> 486166 T13376 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 486166 T13378 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 486166 T13392 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 486174 T13393 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 486174 T13393 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 486175 T13378 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 486175 T13369 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 486175 T13377 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 486175 T13393 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 486177 T13366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 486178 T13366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45041/twi",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:45041_twi",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 486178 T13366 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 486178 T13366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 486180 T13377 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 486180 T13376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 486181 T13392 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 486181 T13377 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 487176 T13393 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 487177 T13393 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 487178 T13393 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 487178 T13393 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 487179 T13393 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 487179 T13393 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/'
[junit4] 2> 487180 T13393 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 487181 T13393 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 487181 T13393 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 487243 T13393 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 487287 T13393 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 487388 T13393 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 487401 T13393 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 487824 T13393 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 487825 T13393 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 487825 T13393 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 487840 T13393 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 487843 T13393 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 487866 T13393 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487871 T13393 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 487875 T13393 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487876 T13393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 487877 T13393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 487877 T13393 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 487878 T13393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 487878 T13393 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 487878 T13393 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 487878 T13393 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 487879 T13393 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/data/
[junit4] 2> 487879 T13393 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@434de008
[junit4] 2> 487880 T13393 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/data
[junit4] 2> 487880 T13393 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/data/index/
[junit4] 2> 487880 T13393 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 487881 T13393 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-C27FC29631BDB979-001/tempDir-003/collection1/data/index
[junit4] 2> 487881 T13393 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=32, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6061597615635959]
[junit4] 2> 487882 T13393 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5e6a6e19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65666d6c),segFN=segments_1,generation=1}
[junit4] 2> 487882 T13393 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 487891 T13393 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 487891 T13393 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 487891 T13393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 487892 T13393 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 487892 T13393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 487892 T13393 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 487893 T13393 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 487893 T13393 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 487893 T13393 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 487894 T13393 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 487895 T13393 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 487895 T13393 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 487896 T13393 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 487896 T13393 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 487897 T13393 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 487898 T13393 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 487917 T13393 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 487922 T13393 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 487922 T13393 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 487924 T13393 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=34.5966796875, floorSegmentMB=2.1396484375, forceMergeDeletesPctAllowed=29.483226279920693, segmentsPerTier=9.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 487924 T13393 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@5e6a6e19 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65666d6c),segFN=segments_1,generation=1}
[junit4] 2> 487925 T13393 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 487925 T13393 oass.SolrIndexSearcher.<init> Opening Searcher@2c0f1a2a[collection1] main
[junit4] 2> 487925 T13393 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 487926 T13393 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 487927 T13393 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 487927 T13393 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 487928 T13393 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 487928 T13393 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 487929 T13393 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Rea
[...truncated too long message...]
37abbf[collection1] main
[junit4] 2> 976754 T15771 C723 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 976754 T15773 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@637abbf[collection1] main{StandardDirectoryReader(segments_ci:1071:nrt _eg(4.10):C11 _eh(4.10):C4)}
[junit4] 2> 976754 T15771 C723 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 56
[junit4] 2> 976756 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=[0,2147483647]&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=0
[junit4] 2> 976759 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=[0,1]&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=1
[junit4] 2> 976761 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=[0,2]&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=0
[junit4] 2> 976763 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=(1,2)&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=0
[junit4] 2> 976766 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=(1,1)&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=1
[junit4] 2> 976768 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=(4,7)&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=0
[junit4] 2> 976770 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={facet=true&f.test_d_dv.facet.interval.set=(123,*)&q=*:*&facet.interval=test_d_dv} hits=15 status=0 QTime=0
[junit4] 2> 976773 T15771 C723 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 976773 T15771 C723 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:*} 0 0
[junit4] 2> 976774 T15771 C723 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 976775 T15771 C723 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fb76b6d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@61895f06),segFN=segments_ci,generation=450}
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@1fb76b6d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@61895f06),segFN=segments_cj,generation=451}
[junit4] 2> 976775 T15771 C723 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 451
[junit4] 2> 976776 T15771 C723 oass.SolrIndexSearcher.<init> Opening Searcher@58142678[collection1] main
[junit4] 2> 976777 T15771 C723 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 976777 T15773 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58142678[collection1] main{StandardDirectoryReader(segments_cj:1072:nrt)}
[junit4] 2> 976777 T15771 C723 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {commit=} 0 3
[junit4] 2> 976779 T15771 C723 oasu.DirectUpdateHandler2.commit start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 976779 T15771 C723 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4] 2> 976780 T15771 C723 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
[junit4] 2> 976780 T15771 C723 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 976780 T15771 C723 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {optimize=} 0 1
[junit4] 2> 976781 T15771 C723 oasc.SolrCore.execute [collection1] webapp=null path=null params={qt=standard&q=*:*&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 976783 T15771 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleFields
[junit4] 2> 976784 T15771 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 976784 T15771 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1623826232
[junit4] 2> 976785 T15771 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@67e665e4
[junit4] 2> 976785 T15771 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=450,autocommits=0,soft autocommits=0,optimizes=18,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=20209,cumulative_deletesById=199,cumulative_deletesByQuery=19,cumulative_errors=0}
[junit4] 2> 976785 T15771 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 976785 T15771 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 976786 T15771 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 976787 T15771 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 976787 T15771 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 976787 T15771 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001 [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001;done=false>>]
[junit4] 2> 976787 T15771 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001
[junit4] 2> 976788 T15771 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001/index;done=false>>]
[junit4] 2> 976788 T15771 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001/init-core-data-001/index
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.request.TestIntervalFaceting-C27FC29631BDB979-001
[junit4] 2> NOTE: test params are: codec=Lucene46, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=pl, timezone=Etc/GMT+11
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=177600952,total=520093696
[junit4] 2> NOTE: All tests run in this JVM: [TestDistributedMissingSort, TestDistributedSearch, TestStressVersions, TestSearchPerf, BadComponentTest, TestReRankQParserPlugin, TestRequestStatusCollectionAPI, JSONWriterTest, MultiThreadedOCPTest, HdfsChaosMonkeySafeLeaderTest, TestCloudManagedSchema, EchoParamsTest, TestCustomSort, ClusterStateUpdateTest, TestBinaryResponseWriter, TestComplexPhraseQParserPlugin, HighlighterTest, WordBreakSolrSpellCheckerTest, BasicDistributedZkTest, SolrTestCaseJ4Test, StatelessScriptUpdateProcessorFactoryTest, CursorPagingTest, TestManagedSchema, DistributedDebugComponentTest, TestNRTOpen, TestIntervalFaceting]
[junit4] Completed on J0 in 129.66s, 17 tests, 1 failure <<< FAILURES!
[...truncated 1150 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 411 suites, 1705 tests, 1 error, 2 failures, 50 ignored (8 assumptions)
Total time: 253 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 617692161 bytes
Compression is 0.0%
Took 1 min 36 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 594 - Still Failing
Posted by Michael McCandless <lu...@mikemccandless.com>.
I committed a fix ...
Mike McCandless
http://blog.mikemccandless.com
On Sat, Aug 9, 2014 at 11:39 AM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/594/
>
> 1 tests failed.
> REGRESSION: org.apache.lucene.index.TestFlushByRamOrCountsPolicy.testFlushDocCount
>
> Error Message:
> Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
>
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
> Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
> at __randomizedtesting.SeedInfo.seed([A89700DEDEDF8395]:0)
> at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:331)
> Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
> at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
> Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
> at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
> ... 8 more
> Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
> at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
> at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
> ... 8 more
>
>
>
>
> Build Log:
> [...truncated 1120 lines...]
> [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
> [junit4] 1> FAILED exc:
> [junit4] 1> java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
> [junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
> [junit4] 1> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
> [junit4] 1> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
> [junit4] 1> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
> [junit4] 1> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
> [junit4] 1> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
> [junit4] 1> at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
> [junit4] 1> Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
> [junit4] 1> at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
> [junit4] 1> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
> [junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
> [junit4] 1> ... 8 more
> [junit4] 1> Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
> [junit4] 1> at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
> [junit4] 1> at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
> [junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
> [junit4] 1> ... 8 more
> [junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFlushByRamOrCountsPolicy -Dtests.method=testFlushDocCount -Dtests.seed=A89700DEDEDF8395 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_EC -Dtests.timezone=US/Michigan -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 0.54s J0 | TestFlushByRamOrCountsPolicy.testFlushDocCount <<<
> [junit4] > Throwable #1: java.lang.AssertionError: expected:<170> but was:<101>
> [junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy.testFlushDocCount(TestFlushByRamOrCountsPolicy.java:160)
> [junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
> [junit4] > Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
> [junit4] > at __randomizedtesting.SeedInfo.seed([A89700DEDEDF8395]:0)
> [junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:331)
> [junit4] > Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
> [junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
> [junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
> [junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
> [junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
> [junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
> [junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
> [junit4] > Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
> [junit4] > at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
> [junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
> [junit4] > ... 8 more
> [junit4] > Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
> [junit4] > at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
> [junit4] > at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
> [junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
> [junit4] > ... 8 more
> [junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build/core/test/J0/./lucene.index.TestFlushByRamOrCountsPolicy-A89700DEDEDF8395-001
> [junit4] 2> NOTE: test params are: codec=Lucene49: {titleTokenized=PostingsFormat(name=Memory doPackFST= false), date=Pulsing41(freqCutoff=9 minBlockSize=35 maxBlockSize=138), body=Pulsing41(freqCutoff=9 minBlockSize=35 maxBlockSize=138), title=Pulsing41(freqCutoff=15 minBlockSize=35 maxBlockSize=138), docid=FSTOrd41}, docValues:{titleDV=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {titleTokenized=DFR G1, body=DFR I(n)L1}, locale=es_EC, timezone=US/Michigan
> [junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=213787512,total=457703424
> [junit4] 2> NOTE: All tests run in this JVM: [TestSimilarityProvider, TestSearch, TestLockFactory, TestIndexWriterCommit, TestLazyProxSkipping, TestCompiledAutomaton, TestTermsEnum, ThrowInUncaught, TestFilteredSearch, TestSloppyPhraseQuery, TestScoreCachingWrappingScorer, TestNumericDocValuesUpdates, TestAllFilesHaveCodecHeader, TestDemo, TestNoDeletionPolicy, TestCharTermAttributeImpl, TestMultiValuedNumericRangeQuery, TestNearSpansOrdered, TestDeletionPolicy, TestBytesStore, TestScorerPerf, Nested, Nested, TestOmitPositions, TestDocTermOrdsRewriteMethod, TestFieldsReader, TestTwoPhaseCommitTool, TestSearcherManager, TestNoMergeScheduler, TestSortRescorer, TestDoc, TestQueryWrapperFilter, TestSearchAfter, TestSort, Test2BPostings, TestFieldCacheRewriteMethod, TestLock, TestSimpleAttributeImpl, TestVirtualMethod, TestBufferedChecksum, TestBytesRef, TestFieldReuse, TestPForDeltaDocIdSet, TestIndexableBinaryStringTools, TestIndexableField, TestEarlyTermination, TestPriorityQueue, TestIndexSearcher, TestPerSegmentDeletes, TestMultiPhraseQuery, TestIndexWriterReader, TestNumericRangeQuery32, TestLucene42DocValuesFormat, TestDocValuesWithThreads, TestLookaheadTokenFilter, TestForceMergeForever, TestIndexCommit, TestTermScorer, Test2BBinaryDocValues, TestIndexWriterConfig, TestIntroSorter, TestLiveFieldValues, TestPrefixRandom, TestMathUtil, TestMultiTermQueryRewrites, TestSpans, TestBagOfPostings, TestTermsEnum2, TestFastDecompressionMode, TestFilterIterator, TestIndexWriterExceptions2, TestStoredFieldsFormat, TestSentinelIntSet, TestTopScoreDocCollector, TestStressNRT, TestReuseDocsEnum, TestMergeSchedulerExternal, Test2BPostingsBytes, TestDocIdBitSet, TestIndexReaderClose, TestExceedMaxTermLength, TestTerms, TestPersistentSnapshotDeletionPolicy, TestPhraseQuery, TestUTF32ToUTF8, TestComplexExplanations, TestLucene45DocValuesFormat, TestSpanExplanations, TestQueryRescorer, TestTopDocsCollector, TestTopFieldCollector, TestReusableStringReader, TestSetOnce, Test2BPagedBytes, TestPostingsFormat, TestPhrasePrefixQuery, TestBinaryDocValuesUpdates, TestByteSlices, TestCrash, TestLongBitSet, TestTermVectors, TestCompoundFile, TestIndexWriterForceMerge, TestIndexWriterLockRelease, TestPayloadsOnVectors, TestRecyclingByteBlockAllocator, TestBlockPostingsFormat3, TestFlushByRamOrCountsPolicy]
> [junit4] Completed on J0 in 10.63s, 5 tests, 1 error <<< FAILURES!
>
> [...truncated 719 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:481: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:454: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build.xml:49: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 417 suites, 3600 tests, 1 error, 129 ignored (118 assumptions)
>
> Total time: 88 minutes 55 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
> Archived 3 artifacts
> Archive block size is 32768
> Received 0 blocks and 4826209 bytes
> Compression is 0.0%
> Took 3.1 sec
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 594 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/594/
1 tests failed.
REGRESSION: org.apache.lucene.index.TestFlushByRamOrCountsPolicy.testFlushDocCount
Error Message:
Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
at __randomizedtesting.SeedInfo.seed([A89700DEDEDF8395]:0)
at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:331)
Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
... 8 more
Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
... 8 more
Build Log:
[...truncated 1120 lines...]
[junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
[junit4] 1> FAILED exc:
[junit4] 1> java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
[junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
[junit4] 1> at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
[junit4] 1> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
[junit4] 1> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
[junit4] 1> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
[junit4] 1> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
[junit4] 1> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
[junit4] 1> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
[junit4] 1> at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
[junit4] 1> Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
[junit4] 1> at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
[junit4] 1> at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
[junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
[junit4] 1> ... 8 more
[junit4] 1> Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
[junit4] 1> at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
[junit4] 1> at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
[junit4] 1> at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
[junit4] 1> ... 8 more
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFlushByRamOrCountsPolicy -Dtests.method=testFlushDocCount -Dtests.seed=A89700DEDEDF8395 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_EC -Dtests.timezone=US/Michigan -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.54s J0 | TestFlushByRamOrCountsPolicy.testFlushDocCount <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected:<170> but was:<101>
[junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy.testFlushDocCount(TestFlushByRamOrCountsPolicy.java:160)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3560, name=Thread-2881, state=RUNNABLE, group=TGRP-TestFlushByRamOrCountsPolicy]
[junit4] > Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
[junit4] > at __randomizedtesting.SeedInfo.seed([A89700DEDEDF8395]:0)
[junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:331)
[junit4] > Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="body" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[125, 125, 123, 123, 123, 123, 123, 115, 117, 98, 115, 116, 99, 124, 125, 125, 125, 123, 123, 123, 49, 125, 125, 125, 124, 123, 123, 123, 112, 49]...', original message: bytes can be at most 32766 in length; got 94384
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:671)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:342)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:301)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:451)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1539)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1254)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1235)
[junit4] > at org.apache.lucene.index.TestFlushByRamOrCountsPolicy$IndexThread.run(TestFlushByRamOrCountsPolicy.java:316)
[junit4] > Caused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 94384
[junit4] > at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)
[junit4] > at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:645)
[junit4] > ... 8 more
[junit4] > Suppressed: java.lang.AssertionError: close() called in wrong state: INCREMENT
[junit4] > at org.apache.lucene.analysis.MockTokenizer.close(MockTokenizer.java:262)
[junit4] > at org.apache.lucene.analysis.TokenFilter.close(TokenFilter.java:58)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:661)
[junit4] > ... 8 more
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build/core/test/J0/./lucene.index.TestFlushByRamOrCountsPolicy-A89700DEDEDF8395-001
[junit4] 2> NOTE: test params are: codec=Lucene49: {titleTokenized=PostingsFormat(name=Memory doPackFST= false), date=Pulsing41(freqCutoff=9 minBlockSize=35 maxBlockSize=138), body=Pulsing41(freqCutoff=9 minBlockSize=35 maxBlockSize=138), title=Pulsing41(freqCutoff=15 minBlockSize=35 maxBlockSize=138), docid=FSTOrd41}, docValues:{titleDV=DocValuesFormat(name=Memory)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {titleTokenized=DFR G1, body=DFR I(n)L1}, locale=es_EC, timezone=US/Michigan
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=213787512,total=457703424
[junit4] 2> NOTE: All tests run in this JVM: [TestSimilarityProvider, TestSearch, TestLockFactory, TestIndexWriterCommit, TestLazyProxSkipping, TestCompiledAutomaton, TestTermsEnum, ThrowInUncaught, TestFilteredSearch, TestSloppyPhraseQuery, TestScoreCachingWrappingScorer, TestNumericDocValuesUpdates, TestAllFilesHaveCodecHeader, TestDemo, TestNoDeletionPolicy, TestCharTermAttributeImpl, TestMultiValuedNumericRangeQuery, TestNearSpansOrdered, TestDeletionPolicy, TestBytesStore, TestScorerPerf, Nested, Nested, TestOmitPositions, TestDocTermOrdsRewriteMethod, TestFieldsReader, TestTwoPhaseCommitTool, TestSearcherManager, TestNoMergeScheduler, TestSortRescorer, TestDoc, TestQueryWrapperFilter, TestSearchAfter, TestSort, Test2BPostings, TestFieldCacheRewriteMethod, TestLock, TestSimpleAttributeImpl, TestVirtualMethod, TestBufferedChecksum, TestBytesRef, TestFieldReuse, TestPForDeltaDocIdSet, TestIndexableBinaryStringTools, TestIndexableField, TestEarlyTermination, TestPriorityQueue, TestIndexSearcher, TestPerSegmentDeletes, TestMultiPhraseQuery, TestIndexWriterReader, TestNumericRangeQuery32, TestLucene42DocValuesFormat, TestDocValuesWithThreads, TestLookaheadTokenFilter, TestForceMergeForever, TestIndexCommit, TestTermScorer, Test2BBinaryDocValues, TestIndexWriterConfig, TestIntroSorter, TestLiveFieldValues, TestPrefixRandom, TestMathUtil, TestMultiTermQueryRewrites, TestSpans, TestBagOfPostings, TestTermsEnum2, TestFastDecompressionMode, TestFilterIterator, TestIndexWriterExceptions2, TestStoredFieldsFormat, TestSentinelIntSet, TestTopScoreDocCollector, TestStressNRT, TestReuseDocsEnum, TestMergeSchedulerExternal, Test2BPostingsBytes, TestDocIdBitSet, TestIndexReaderClose, TestExceedMaxTermLength, TestTerms, TestPersistentSnapshotDeletionPolicy, TestPhraseQuery, TestUTF32ToUTF8, TestComplexExplanations, TestLucene45DocValuesFormat, TestSpanExplanations, TestQueryRescorer, TestTopDocsCollector, TestTopFieldCollector, TestReusableStringReader, TestSetOnce, Test2BPagedBytes, TestPostingsFormat, TestPhrasePrefixQuery, TestBinaryDocValuesUpdates, TestByteSlices, TestCrash, TestLongBitSet, TestTermVectors, TestCompoundFile, TestIndexWriterForceMerge, TestIndexWriterLockRelease, TestPayloadsOnVectors, TestRecyclingByteBlockAllocator, TestBlockPostingsFormat3, TestFlushByRamOrCountsPolicy]
[junit4] Completed on J0 in 10.63s, 5 tests, 1 error <<< FAILURES!
[...truncated 719 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:481: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/build.xml:49: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 417 suites, 3600 tests, 1 error, 129 ignored (118 assumptions)
Total time: 88 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 4826209 bytes
Compression is 0.0%
Took 3.1 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 593 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/593/
1 tests failed.
REGRESSION: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testDistribSearch
Error Message:
IOException occured when talking to server at: http://127.0.0.1:30047/_sd/ya/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when talking to server at: http://127.0.0.1:30047/_sd/ya/collection1
at __randomizedtesting.SeedInfo.seed([2F90961CE3B07E66:AE76180494EF1E5A]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:563)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
at org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:91)
at org.apache.solr.client.solrj.SolrServer.query(SolrServer.java:301)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:223)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:165)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrServer(FullSolrCloudDistribCmdsTest.java:414)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.doTest(FullSolrCloudDistribCmdsTest.java:144)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:159)
at org.apache.http.impl.io.AbstractSessionOutputBuffer.writeLine(AbstractSessionOutputBuffer.java:258)
at org.apache.http.impl.io.HttpRequestWriter.writeHeadLine(HttpRequestWriter.java:59)
at org.apache.http.impl.io.HttpRequestWriter.writeHeadLine(HttpRequestWriter.java:46)
at org.apache.http.impl.io.AbstractMessageWriter.write(AbstractMessageWriter.java:109)
at org.apache.http.impl.AbstractHttpClientConnection.sendRequestHeader(AbstractHttpClientConnection.java:254)
at org.apache.http.impl.conn.DefaultClientConnection.sendRequestHeader(DefaultClientConnection.java:270)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestHeader(ManagedClientConnectionImpl.java:209)
at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:203)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:121)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:449)
... 50 more
Build Log:
[...truncated 14190 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/init-core-data-001
[junit4] 2> 3858921 T11953 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_sd/ya
[junit4] 2> 3858924 T11953 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 3858925 T11953 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3858926 T11954 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 3859026 T11953 oasc.ZkTestServer.run start zk server on port:53895
[junit4] 2> 3859027 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3859031 T11960 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11afc564 name:ZooKeeperConnection Watcher:127.0.0.1:53895 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3859031 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3859031 T11953 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 3859035 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3859036 T11962 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6bdf7d45 name:ZooKeeperConnection Watcher:127.0.0.1:53895/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3859036 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3859036 T11953 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 3859038 T11953 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 3859039 T11953 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 3859040 T11953 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 3859042 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3859042 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 3859044 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 3859045 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 3859147 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3859148 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3859150 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3859150 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 3859152 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3859153 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 3859154 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3859155 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 3859157 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3859157 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 3859159 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3859160 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 3859162 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3859162 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3859164 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3859165 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 3859166 T11953 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3859167 T11953 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 3860300 T11953 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3860305 T11953 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30047
[junit4] 2> 3860305 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3860306 T11953 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3860306 T11953 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002
[junit4] 2> 3860306 T11953 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/'
[junit4] 2> 3860336 T11953 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/solr.xml
[junit4] 2> 3860387 T11953 oasc.CoreContainer.<init> New CoreContainer 990570062
[junit4] 2> 3860387 T11953 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/]
[junit4] 2> 3860388 T11953 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3860388 T11953 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3860388 T11953 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3860389 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3860389 T11953 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3860389 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3860389 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3860390 T11953 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3860390 T11953 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3860390 T11953 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3860391 T11953 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3860391 T11953 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3860391 T11953 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3860391 T11953 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53895/solr
[junit4] 2> 3860392 T11953 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3860392 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3860394 T11973 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4896292a name:ZooKeeperConnection Watcher:127.0.0.1:53895 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3860394 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3860397 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3860398 T11975 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f926a08 name:ZooKeeperConnection Watcher:127.0.0.1:53895/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3860398 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3860400 T11953 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 3860402 T11953 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 3860403 T11953 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 3860405 T11953 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 3860407 T11953 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 3860409 T11953 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 3860410 T11953 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30047__sd%2Fya
[junit4] 2> 3860411 T11953 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30047__sd%2Fya
[junit4] 2> 3860412 T11953 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 3860414 T11953 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 3860416 T11953 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:30047__sd%2Fya
[junit4] 2> 3860416 T11953 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 3860417 T11953 oasc.Overseer.start Overseer (id=92242611182895107-127.0.0.1:30047__sd%2Fya-n_0000000000) starting
[junit4] 2> 3860419 T11953 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 3860424 T11977 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 3860424 T11953 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 3860426 T11953 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 3860427 T11953 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3860429 T11976 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 3860434 T11978 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 3860434 T11978 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3860435 T11978 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3860435 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3860436 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3860436 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:30047/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:30047__sd%2Fya",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3860437 T11976 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 3860437 T11976 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3860439 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3861435 T11978 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3861436 T11978 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 3861436 T11978 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3861437 T11978 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3861437 T11978 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3861438 T11978 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/collection1/'
[junit4] 2> 3861440 T11978 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 3861440 T11978 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 3861441 T11978 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 3861503 T11978 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3861546 T11978 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3861647 T11978 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3861656 T11978 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3862023 T11978 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3862026 T11978 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3862028 T11978 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3862032 T11978 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3862048 T11978 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 3862048 T11978 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3862049 T11978 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-002/collection1/, dataDir=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/control/data/
[junit4] 2> 3862049 T11978 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@290f6b0c
[junit4] 2> 3862051 T11978 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/control/data
[junit4] 2> 3862051 T11978 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/control/data/index/
[junit4] 2> 3862052 T11978 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 3862053 T11978 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/control/data/index
[junit4] 2> 3862053 T11978 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3862056 T11978 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fb9c1a),segFN=segments_1,generation=1}
[junit4] 2> 3862056 T11978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3862059 T11978 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3862059 T11978 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3862060 T11978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3862060 T11978 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3862060 T11978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3862060 T11978 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3862061 T11978 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3862061 T11978 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3862061 T11978 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3862062 T11978 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3862062 T11978 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3862063 T11978 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3862063 T11978 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3862064 T11978 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3862064 T11978 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3862064 T11978 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3862064 T11978 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3862065 T11978 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3862065 T11978 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3862065 T11978 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3862066 T11978 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3862085 T11978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3862087 T11978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3862088 T11978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3862090 T11978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3862092 T11978 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3862098 T11978 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3862098 T11978 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3862099 T11978 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1470702014, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 3862101 T11978 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-001 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fb9c1a),segFN=segments_1,generation=1}
[junit4] 2> 3862101 T11978 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3862101 T11978 oass.SolrIndexSearcher.<init> Opening Searcher@7e4478[collection1] main
[junit4] 2> 3862102 T11978 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3862102 T11978 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3862103 T11978 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3862103 T11978 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3862103 T11978 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3862104 T11978 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3862104 T11978 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 3862104 T11978 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3862105 T11978 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 3862106 T11978 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 3862107 T11978 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3862107 T11978 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3862109 T11980 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7e4478[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3862111 T11978 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3862111 T11983 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30047/_sd/ya collection:control_collection shard:shard1
[junit4] 2> 3862112 T11953 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1
[junit4] 2> 3862112 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3862112 T11983 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 3862113 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3862115 T11985 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c5a71b3 name:ZooKeeperConnection Watcher:127.0.0.1:53895/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3862115 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3862118 T11953 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3862119 T11983 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3862119 T11953 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3862120 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3862120 T11983 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3862121 T11983 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C11446 name=collection1 org.apache.solr.core.SolrCore@648819c8 url=http://127.0.0.1:30047/_sd/ya/collection1 node=127.0.0.1:30047__sd%2Fya C11446_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:30047/_sd/ya, core=collection1, node_name=127.0.0.1:30047__sd%2Fya}
[junit4] 2> 3862121 T11983 C11446 P30047 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:30047/_sd/ya/collection1/
[junit4] 2> 3862121 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3862121 T11983 C11446 P30047 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3862122 T11983 C11446 P30047 oasc.SyncStrategy.syncToMe http://127.0.0.1:30047/_sd/ya/collection1/ has no replicas
[junit4] 2> 3862122 T11983 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:30047/_sd/ya/collection1/ shard1
[junit4] 2> 3862122 T11983 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 3862123 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3862124 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3862126 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3862127 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3862230 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3862230 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3862279 T11983 oasc.ZkController.register We are http://127.0.0.1:30047/_sd/ya/collection1/ and leader is http://127.0.0.1:30047/_sd/ya/collection1/
[junit4] 2> 3862280 T11983 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30047/_sd/ya
[junit4] 2> 3862280 T11983 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3862280 T11983 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 3862280 T11983 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3862281 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3862282 T11983 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3862282 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3862283 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:30047/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:30047__sd%2Fya",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3862386 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3862386 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3863266 T11953 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 3863267 T11953 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3863270 T11953 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:59459
[junit4] 2> 3863270 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3863271 T11953 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3863271 T11953 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003
[junit4] 2> 3863271 T11953 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/'
[junit4] 2> 3863301 T11953 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/solr.xml
[junit4] 2> 3863351 T11953 oasc.CoreContainer.<init> New CoreContainer 2092249032
[junit4] 2> 3863352 T11953 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/]
[junit4] 2> 3863353 T11953 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3863353 T11953 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3863353 T11953 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3863353 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3863354 T11953 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3863354 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3863354 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3863354 T11953 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3863355 T11953 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3863355 T11953 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3863355 T11953 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3863356 T11953 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3863356 T11953 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3863356 T11953 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53895/solr
[junit4] 2> 3863356 T11953 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3863357 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3863358 T11997 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b54336e name:ZooKeeperConnection Watcher:127.0.0.1:53895 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3863359 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3863362 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3863362 T11999 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15e4cea6 name:ZooKeeperConnection Watcher:127.0.0.1:53895/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3863363 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3863367 T11953 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3864370 T11953 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59459__sd%2Fya
[junit4] 2> 3864371 T11953 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59459__sd%2Fya
[junit4] 2> 3864374 T11979 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3864375 T12000 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3864374 T11986 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3864382 T12001 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3864382 T12001 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3864383 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3864383 T12001 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3864384 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3864385 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59459/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59459__sd%2Fya",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3864385 T11976 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 3864385 T11976 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 3864387 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3864387 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3864387 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3865384 T12001 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3865385 T12001 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3865385 T12001 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3865386 T12001 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3865386 T12001 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3865387 T12001 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/collection1/'
[junit4] 2> 3865389 T12001 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 3865389 T12001 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 3865390 T12001 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 3865466 T12001 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3865509 T12001 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3865611 T12001 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3865620 T12001 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3865984 T12001 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3865987 T12001 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3865989 T12001 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3865993 T12001 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3866010 T12001 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3866010 T12001 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3866011 T12001 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-003/collection1/, dataDir=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty1/
[junit4] 2> 3866011 T12001 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@290f6b0c
[junit4] 2> 3866013 T12001 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty1
[junit4] 2> 3866014 T12001 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty1/index/
[junit4] 2> 3866014 T12001 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 3866017 T12001 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty1/index
[junit4] 2> 3866017 T12001 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3866020 T12001 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3081f390),segFN=segments_1,generation=1}
[junit4] 2> 3866020 T12001 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3866023 T12001 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3866024 T12001 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3866024 T12001 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3866024 T12001 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3866025 T12001 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3866025 T12001 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3866025 T12001 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3866026 T12001 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3866026 T12001 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3866027 T12001 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3866027 T12001 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3866027 T12001 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3866028 T12001 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3866028 T12001 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3866029 T12001 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3866029 T12001 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3866029 T12001 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3866030 T12001 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3866030 T12001 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3866030 T12001 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3866031 T12001 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3866050 T12001 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3866052 T12001 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3866054 T12001 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3866056 T12001 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3866057 T12001 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3866062 T12001 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3866062 T12001 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3866064 T12001 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1470702014, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 3866066 T12001 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-002 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3081f390),segFN=segments_1,generation=1}
[junit4] 2> 3866066 T12001 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3866066 T12001 oass.SolrIndexSearcher.<init> Opening Searcher@39d3be2b[collection1] main
[junit4] 2> 3866067 T12001 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3866068 T12001 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3866068 T12001 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3866068 T12001 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3866068 T12001 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3866069 T12001 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3866069 T12001 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3866070 T12001 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3866070 T12001 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3866070 T12001 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3866073 T12002 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39d3be2b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3866076 T12001 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3866077 T12005 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59459/_sd/ya collection:collection1 shard:shard3
[junit4] 2> 3866077 T11953 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1
[junit4] 2> 3866077 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3866077 T12005 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 3866082 T12005 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard3
[junit4] 2> 3866083 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3866083 T12005 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3866084 T12005 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C11447 name=collection1 org.apache.solr.core.SolrCore@828538d url=http://127.0.0.1:59459/_sd/ya/collection1 node=127.0.0.1:59459__sd%2Fya C11447_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:59459/_sd/ya, core=collection1, node_name=127.0.0.1:59459__sd%2Fya}
[junit4] 2> 3866084 T12005 C11447 P59459 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59459/_sd/ya/collection1/
[junit4] 2> 3866084 T12005 C11447 P59459 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3866084 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3866084 T12005 C11447 P59459 oasc.SyncStrategy.syncToMe http://127.0.0.1:59459/_sd/ya/collection1/ has no replicas
[junit4] 2> 3866085 T12005 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59459/_sd/ya/collection1/ shard3
[junit4] 2> 3866085 T12005 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 3866087 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866087 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866087 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866089 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3866090 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3866193 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866193 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866193 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866242 T12005 oasc.ZkController.register We are http://127.0.0.1:59459/_sd/ya/collection1/ and leader is http://127.0.0.1:59459/_sd/ya/collection1/
[junit4] 2> 3866242 T12005 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59459/_sd/ya
[junit4] 2> 3866242 T12005 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3866242 T12005 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3866242 T12005 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3866243 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3866244 T12005 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3866244 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3866245 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:59459/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:59459__sd%2Fya",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3866348 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866348 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3866348 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3867236 T11953 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 3867237 T11953 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3867240 T11953 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26648
[junit4] 2> 3867241 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3867241 T11953 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3867241 T11953 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004
[junit4] 2> 3867241 T11953 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/'
[junit4] 2> 3867271 T11953 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/solr.xml
[junit4] 2> 3867321 T11953 oasc.CoreContainer.<init> New CoreContainer 2035010764
[junit4] 2> 3867322 T11953 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/]
[junit4] 2> 3867323 T11953 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3867323 T11953 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3867323 T11953 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3867323 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3867324 T11953 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3867324 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3867324 T11953 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3867324 T11953 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3867324 T11953 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3867325 T11953 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3867325 T11953 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3867325 T11953 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3867326 T11953 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3867326 T11953 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:53895/solr
[junit4] 2> 3867326 T11953 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3867327 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3867328 T12016 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f72a34a name:ZooKeeperConnection Watcher:127.0.0.1:53895 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3867329 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3867331 T11953 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3867332 T12018 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10a1c58f name:ZooKeeperConnection Watcher:127.0.0.1:53895/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3867332 T11953 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3867337 T11953 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3868339 T11953 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26648__sd%2Fya
[junit4] 2> 3868340 T11953 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26648__sd%2Fya
[junit4] 2> 3868343 T11979 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3868343 T12019 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3868343 T12000 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3868343 T11986 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3868350 T12020 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3868350 T12020 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3868351 T12020 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3868351 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3868352 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3868353 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:26648/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:26648__sd%2Fya",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3868353 T11976 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 3868353 T11976 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 3868355 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3868355 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3868355 T12019 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3868355 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3869351 T12020 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3869352 T12020 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3869353 T12020 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3869353 T12020 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3869353 T12020 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3869354 T12020 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/collection1/'
[junit4] 2> 3869356 T12020 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 3869357 T12020 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 3869357 T12020 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 3869420 T12020 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3869465 T12020 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3869566 T12020 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3869575 T12020 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3869942 T12020 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3869945 T12020 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3869947 T12020 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3869951 T12020 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3869967 T12020 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3869968 T12020 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3869968 T12020 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-004/collection1/, dataDir=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty2/
[junit4] 2> 3869968 T12020 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@290f6b0c
[junit4] 2> 3869972 T12020 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty2
[junit4] 2> 3869973 T12020 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty2/index/
[junit4] 2> 3869973 T12020 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 3869975 T12020 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/tempDir-001/jetty2/index
[junit4] 2> 3869975 T12020 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3869978 T12020 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@462a284f),segFN=segments_1,generation=1}
[junit4] 2> 3869978 T12020 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3869982 T12020 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3869982 T12020 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3869982 T12020 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3869983 T12020 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3869983 T12020 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3869983 T12020 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3869983 T12020 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3869984 T12020 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3869984 T12020 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3869985 T12020 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3869985 T12020 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3869985 T12020 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3869986 T12020 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3869986 T12020 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3869987 T12020 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3869987 T12020 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3869987 T12020 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3869988 T12020 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3869988 T12020 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3869988 T12020 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3869989 T12020 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3870008 T12020 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3870010 T12020 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3870011 T12020 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3870013 T12020 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3870015 T12020 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3870020 T12020 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3870020 T12020 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3870021 T12020 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1470702014, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 3870023 T12020 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(SimpleFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1/solr.cloud.FullSolrCloudDistribCmdsTest-2F90961CE3B07E66-001/index-SimpleFSDirectory-003 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@462a284f),segFN=segments_1,generation=1}
[junit4] 2> 3870023 T12020 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3870023 T12020 oass.SolrIndexSearcher.<init> Opening Searcher@2c589765[collection1] main
[junit4] 2> 3870024 T12020 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3870024 T12020 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3870025 T12020 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3870025 T12020 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3870025 T12020 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3870026 T12020 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3870026 T12020 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3870027 T12020 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3870027 T12020 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3870027 T12020 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3870030 T12021 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2c589765[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3870032 T12020 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3870033 T12024 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26648/_sd/ya collection:collection1 shard:shard2
[junit4] 2> 3870033 T11953 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J1
[junit4] 2> 3870034 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3870034 T12024 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 3870038 T12024 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 3870039 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3870039 T12024 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3870040 T12024 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C11448 name=collection1 org.apache.solr.core.SolrCore@79773203 url=http://127.0.0.1:26648/_sd/ya/collection1 node=127.0.0.1:26648__sd%2Fya C11448_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:26648/_sd/ya, core=collection1, node_name=127.0.0.1:26648__sd%2Fya}
[junit4] 2> 3870040 T12024 C11448 P26648 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:26648/_sd/ya/collection1/
[junit4] 2> 3870040 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3870040 T12024 C11448 P26648 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3870041 T12024 C11448 P26648 oasc.SyncStrategy.syncToMe http://127.0.0.1:26648/_sd/ya/collection1/ has no replicas
[junit4] 2> 3870041 T12024 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:26648/_sd/ya/collection1/ shard2
[junit4] 2> 3870041 T12024 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 3870042 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870042 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870042 T12019 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870042 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870045 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3870047 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3870047 T12019 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870048 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870048 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870047 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870095 T12024 oasc.ZkController.register We are http://127.0.0.1:26648/_sd/ya/collection1/ and leader is http://127.0.0.1:26648/_sd/ya/collection1/
[junit4] 2> 3870096 T12024 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26648/_sd/ya
[junit4] 2> 3870096 T12024 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3870096 T12024 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3870096 T12024 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3870097 T11979 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3870097 T12024 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3870098 T11976 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3870099 T11976 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:26648/_sd/ya",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:26648__sd%2Fya",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 3870202 T12019 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870202 T11979 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870202 T12000 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3870202 T11986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3871236 T11953 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 3871237 T11953 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3871240 T11953 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14563
[junit4] 2> 3871240 T11953 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3871241 T11953 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3871241 T11953 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hud
[...truncated too long message...]
dPulsing41), 5grand3_s=PostingsFormat(name=Direct), 7grand6_s=PostingsFormat(name=MockRandom), 48parent_f1_s=PostingsFormat(name=Direct), 36parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 4grand2_s=PostingsFormat(name=FSTOrdPulsing41), 7grand4_s=Lucene41(blocksize=128), 44parent_f2_s=Lucene41(blocksize=128), 7grand1_s=PostingsFormat(name=Direct), 2grand1_s=Lucene41(blocksize=128), 16parent_f1_s=Lucene41(blocksize=128), 9parent_f1_s=Lucene41(blocksize=128), 11parent_f2_s=Lucene41(blocksize=128), 23parent_f1_s=PostingsFormat(name=MockRandom), 35parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 3child0_s=Lucene41(blocksize=128), 15parent_f2_s=Lucene41(blocksize=128), 44parent_f1_s=PostingsFormat(name=Direct), 47parent_f1_s=Lucene41(blocksize=128), 3grand2_s=PostingsFormat(name=MockRandom), 4child2_s=PostingsFormat(name=FSTOrdPulsing41), 46parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 24parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 0parent_f1_s=PostingsFormat(name=Direct), 20parent_f2_s=PostingsFormat(name=MockRandom), 7parent_f1_s=PostingsFormat(name=MockRandom), 7grand5_s=PostingsFormat(name=Direct), 48parent_f2_s=Lucene41(blocksize=128), _version_=PostingsFormat(name=FSTOrdPulsing41), 23parent_f2_s=PostingsFormat(name=Direct), 2child0_s=PostingsFormat(name=FSTOrdPulsing41), 16parent_f2_s=PostingsFormat(name=Direct), 21parent_f1_s=Lucene41(blocksize=128), 19parent_f2_s=PostingsFormat(name=MockRandom), 38parent_f2_s=PostingsFormat(name=Direct), 31parent_f2_s=PostingsFormat(name=MockRandom), 51parent_f1_s=PostingsFormat(name=Direct), 5parent_f2_s=PostingsFormat(name=Direct), _root_=PostingsFormat(name=FSTOrdPulsing41), 17parent_f2_s=Lucene41(blocksize=128), 50parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 40parent_f1_s=PostingsFormat(name=Direct), 7grand0_s=Lucene41(blocksize=128), 18parent_f1_s=PostingsFormat(name=MockRandom), 4grand0_s=PostingsFormat(name=Direct), 54parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 28parent_f2_s=Lucene41(blocksize=128), 53parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 6grand4_s=PostingsFormat(name=FSTOrdPulsing41), 22parent_f2_s=Lucene41(blocksize=128), 2grand0_s=PostingsFormat(name=FSTOrdPulsing41), 37parent_f1_s=PostingsFormat(name=Direct), 53parent_f2_s=PostingsFormat(name=MockRandom), 49parent_f2_s=PostingsFormat(name=Direct), 24parent_f2_s=PostingsFormat(name=MockRandom), 33parent_f2_s=Lucene41(blocksize=128), 34parent_f1_s=PostingsFormat(name=MockRandom), 34parent_f2_s=PostingsFormat(name=Direct), 27parent_f2_s=PostingsFormat(name=Direct), 2child1_s=PostingsFormat(name=MockRandom), 3parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 30parent_f1_s=PostingsFormat(name=MockRandom), 39parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 11parent_f1_s=PostingsFormat(name=Direct), 4child3_s=Lucene41(blocksize=128), 4child0_s=PostingsFormat(name=Direct), 49parent_f1_s=PostingsFormat(name=MockRandom), 14parent_f1_s=Lucene41(blocksize=128), 14parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 32parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 41parent_f1_s=PostingsFormat(name=MockRandom), 9parent_f2_s=PostingsFormat(name=Direct), 29parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 19parent_f1_s=PostingsFormat(name=Direct), 2parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 27parent_f1_s=PostingsFormat(name=MockRandom), 54parent_f1_s=Lucene41(blocksize=128), 18parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 1parent_f1_s=Lucene41(blocksize=128), 40parent_f2_s=Lucene41(blocksize=128), 7parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 5grand2_s=Lucene41(blocksize=128), 51parent_f2_s=Lucene41(blocksize=128), 12parent_f2_s=PostingsFormat(name=Direct), 5grand1_s=PostingsFormat(name=FSTOrdPulsing41), 22parent_f1_s=PostingsFormat(name=Direct), 3grand1_s=PostingsFormat(name=Direct), type_s=PostingsFormat(name=FSTOrdPulsing41), 50parent_f1_s=Lucene41(blocksize=128), 13parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 43parent_f1_s=Lucene41(blocksize=128), 6parent_f2_s=Lucene41(blocksize=128), 46parent_f2_s=PostingsFormat(name=MockRandom), 28parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 1grand0_s=Lucene41(blocksize=128), 4parent_f1_s=PostingsFormat(name=Direct), 8parent_f2_s=PostingsFormat(name=MockRandom), 3parent_f1_s=PostingsFormat(name=MockRandom), 31parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 4child1_s=PostingsFormat(name=MockRandom), 42parent_f2_s=PostingsFormat(name=MockRandom), 38parent_f1_s=PostingsFormat(name=MockRandom), 6parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 30parent_f2_s=PostingsFormat(name=Direct), 6grand0_s=PostingsFormat(name=FSTOrdPulsing41), 5grand4_s=PostingsFormat(name=MockRandom), 4grand3_s=PostingsFormat(name=MockRandom), 33parent_f1_s=PostingsFormat(name=Direct), 6grand1_s=PostingsFormat(name=MockRandom), 1parent_f2_s=PostingsFormat(name=Direct), 45parent_f1_s=PostingsFormat(name=MockRandom), 29parent_f1_s=PostingsFormat(name=MockRandom), 39parent_f2_s=PostingsFormat(name=MockRandom), 4parent_f2_s=PostingsFormat(name=MockRandom), 32parent_f1_s=Lucene41(blocksize=128), 10parent_f1_s=Lucene41(blocksize=128), 35parent_f2_s=PostingsFormat(name=MockRandom), 37parent_f2_s=Lucene41(blocksize=128), 25parent_f1_s=Lucene41(blocksize=128), 7grand3_s=PostingsFormat(name=FSTOrdPulsing41), 21parent_f2_s=PostingsFormat(name=FSTOrdPulsing41), 36parent_f1_s=Lucene41(blocksize=128), text_t=PostingsFormat(name=FSTOrdPulsing41), 17parent_f1_s=PostingsFormat(name=FSTOrdPulsing41), 13parent_f2_s=PostingsFormat(name=MockRandom), 3child2_s=PostingsFormat(name=MockRandom), 4grand1_s=Lucene41(blocksize=128), rnd_b=PostingsFormat(name=MockRandom)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_IN, timezone=Antarctica/Syowa
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=211459184,total=477102080
[junit4] 2> NOTE: All tests run in this JVM: [TestBadConfig, SampleTest, TestCursorMarkWithoutUniqueKey, HdfsRecoveryZkTest, DistributedQueryElevationComponentTest, TermVectorComponentDistributedTest, TestMultiCoreConfBootstrap, SSLMigrationTest, TestDynamicFieldCollectionResource, TestMissingGroups, TestNoOpRegenerator, HttpPartitionTest, DistributedSuggestComponentTest, CacheHeaderTest, TestSweetSpotSimilarityFactory, TestElisionMultitermQuery, AutoCommitTest, TestWriterPerf, SearchHandlerTest, TestFiltering, SOLR749Test, SuggesterFSTTest, TestLMDirichletSimilarityFactory, TestDistributedGrouping, TestUpdate, TestSolrQueryParser, TestManagedSchemaFieldResource, MultiTermTest, HardAutoCommitTest, AnalysisErrorHandlingTest, SoftAutoCommitTest, TestFastWriter, DistributedQueryComponentCustomSortTest, ChaosMonkeyNothingIsSafeTest, LukeRequestHandlerTest, TestPseudoReturnFields, OpenCloseCoreStressTest, DocValuesTest, CircularListTest, LegacyHTMLStripCharFilterTest, DistributedQueryComponentOptimizationTest, BasicDistributedZk2Test, RequestHandlersTest, DocValuesMissingTest, TestDocBasedVersionConstraints, TestQueryUtils, TestValueSourceCache, TestIndexingPerformance, DefaultValueUpdateProcessorTest, TestCSVResponseWriter, ResponseHeaderTest, TestReversedWildcardFilterFactory, CopyFieldTest, ClusterStateTest, HdfsLockFactoryTest, TestRecoveryHdfs, TestCollationFieldDocValues, AddBlockUpdateTest, DOMUtilTest, TestImplicitCoreProperties, BinaryUpdateRequestHandlerTest, TestSolr4Spatial2, TestSystemIdResolver, TestFieldCollectionResource, TestGroupingSearch, TestFastLRUCache, TestInfoStreamLogging, QueryParsingTest, InfoHandlerTest, ZkSolrClientTest, AnalyticsMergeStrategyTest, TestJoin, UniqFieldsUpdateProcessorFactoryTest, TestPerFieldSimilarity, OutputWriterTest, TestPHPSerializedResponseWriter, DirectUpdateHandlerOptimizeTest, TimeZoneUtilsTest, PrimUtilsTest, TestConfigSets, TestIndexSearcher, TestFieldSortValues, ShardSplitTest, DocExpirationUpdateProcessorFactoryTest, TestSerializedLuceneMatchVersion, CoreContainerCoreInitFailuresTest, DeleteReplicaTest, TestStressLucene, EnumFieldTest, TestRandomFaceting, TestCloudInspectUtil, TestAtomicUpdateErrorCases, DistributedTermsComponentTest, TestBlendedInfixSuggestions, TestZkChroot, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed on J1 in 141.49s, 1 test, 1 error <<< FAILURES!
[...truncated 650 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:481: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:454: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 410 suites, 1711 tests, 1 error, 50 ignored (8 assumptions)
Total time: 286 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 461190114 bytes
Compression is 0.0%
Took 1 min 15 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 592 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/592/
2 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
Captured an uncaught exception in thread: Thread[id=780113, name=RecoveryThread, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=780113, name=RecoveryThread, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5]:0)
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
at org.apache.solr.client.solrj.impl.HttpSolrServer.httpUriRequest(HttpSolrServer.java:241)
at org.apache.solr.client.solrj.impl.HttpSolrServer.httpUriRequest(HttpSolrServer.java:230)
at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:610)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:371)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5:2A7C068A85F9EB89]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11458 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/init-core-data-001
[junit4] 2> 230469 T2098 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 230469 T2098 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /a/xe
[junit4] 2> 230474 T2098 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 230475 T2098 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 230476 T2099 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 230576 T2098 oasc.ZkTestServer.run start zk server on port:37975
[junit4] 2> 230578 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 230581 T2105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17c0ae63 name:ZooKeeperConnection Watcher:127.0.0.1:37975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 230582 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 230582 T2098 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 230585 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 230586 T2107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@755629fd name:ZooKeeperConnection Watcher:127.0.0.1:37975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 230586 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 230586 T2098 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 230587 T2098 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 230589 T2098 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 230590 T2098 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 230591 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 230592 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 230594 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 230594 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 230596 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 230597 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 230598 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 230598 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 230600 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 230600 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 230601 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 230607 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 230608 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 230609 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 230610 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 230610 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 230612 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 230612 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 230613 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 230614 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 230615 T2098 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 230615 T2098 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 231663 T2098 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 231667 T2098 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:13902
[junit4] 2> 231668 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 231668 T2098 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 231668 T2098 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002
[junit4] 2> 231669 T2098 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/'
[junit4] 2> 231698 T2098 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/solr.xml
[junit4] 2> 231752 T2098 oasc.CoreContainer.<init> New CoreContainer 1185402054
[junit4] 2> 231753 T2098 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/]
[junit4] 2> 231754 T2098 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 231754 T2098 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 231754 T2098 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 231754 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 231755 T2098 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 231755 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 231755 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 231755 T2098 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 231756 T2098 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 231756 T2098 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 231756 T2098 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 231757 T2098 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 231757 T2098 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 231757 T2098 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37975/solr
[junit4] 2> 231757 T2098 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 231758 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 231759 T2118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3274ff67 name:ZooKeeperConnection Watcher:127.0.0.1:37975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 231760 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 231761 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 231762 T2120 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a8e394c name:ZooKeeperConnection Watcher:127.0.0.1:37975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 231762 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 231763 T2098 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 231765 T2098 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 231767 T2098 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 231769 T2098 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 231770 T2098 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 231772 T2098 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 231773 T2098 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:13902_a%2Fxe
[junit4] 2> 231774 T2098 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:13902_a%2Fxe
[junit4] 2> 231775 T2098 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 231777 T2098 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 231779 T2098 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:13902_a%2Fxe
[junit4] 2> 231779 T2098 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 231781 T2098 oasc.Overseer.start Overseer (id=92232553437659139-127.0.0.1:13902_a%2Fxe-n_0000000000) starting
[junit4] 2> 231783 T2098 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 231788 T2122 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 231788 T2098 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 231790 T2098 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 231791 T2098 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 231793 T2121 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 231798 T2123 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 231799 T2123 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 231799 T2123 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 231800 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 231801 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 231801 T2121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:13902/a/xe",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:13902_a%2Fxe",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 231801 T2121 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 231801 T2121 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 231803 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 231804 T2124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 232800 T2123 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 232801 T2123 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 232801 T2123 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 232802 T2123 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 232802 T2123 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 232803 T2123 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/collection1/'
[junit4] 2> 232805 T2123 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 232805 T2123 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 232806 T2123 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 232873 T2123 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 232922 T2123 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 232923 T2123 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 232938 T2123 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 233394 T2123 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 233395 T2123 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 233395 T2123 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 233410 T2123 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 233413 T2123 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 233433 T2123 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 233438 T2123 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 233442 T2123 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 233443 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 233443 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 233443 T2123 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 233444 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 233444 T2123 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 233445 T2123 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 233445 T2123 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 233445 T2123 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-002/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-001/control/data/
[junit4] 2> 233445 T2123 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f8c26ff
[junit4] 2> 233456 T2123 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-001/control/data
[junit4] 2> 233457 T2123 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-001/control/data/index/
[junit4] 2> 233458 T2123 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 233458 T2123 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-001/control/data/index
[junit4] 2> 233458 T2123 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=1.2470703125, floorSegmentMB=2.1884765625, forceMergeDeletesPctAllowed=23.277766949976346, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 233459 T2123 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@6b88d968 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67ea74f),segFN=segments_1,generation=1}
[junit4] 2> 233460 T2123 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 233475 T2123 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 233475 T2123 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 233476 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 233476 T2123 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 233476 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 233476 T2123 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 233477 T2123 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 233478 T2123 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 233478 T2123 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 233478 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 233479 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 233479 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 233480 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 233480 T2123 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 233481 T2123 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 233481 T2123 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 233482 T2123 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 233482 T2123 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 233483 T2123 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 233483 T2123 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 233484 T2123 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 233503 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 233505 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 233507 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 233508 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 233510 T2123 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 233516 T2123 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 233516 T2123 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 233517 T2123 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=90.3037109375, floorSegmentMB=0.8642578125, forceMergeDeletesPctAllowed=12.1405079704558, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 233518 T2123 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@6b88d968 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@67ea74f),segFN=segments_1,generation=1}
[junit4] 2> 233518 T2123 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 233519 T2123 oass.SolrIndexSearcher.<init> Opening Searcher@29423654[collection1] main
[junit4] 2> 233519 T2123 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 233520 T2123 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 233520 T2123 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 233521 T2123 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 233521 T2123 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 233521 T2123 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 233522 T2123 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 233522 T2123 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 233522 T2123 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 233524 T2123 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 233524 T2123 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 233525 T2123 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 233528 T2125 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@29423654[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 233532 T2123 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 233533 T2128 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:13902/a/xe collection:control_collection shard:shard1
[junit4] 2> 233534 T2098 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 233534 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 233534 T2128 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 233535 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 233537 T2130 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45d5e97e name:ZooKeeperConnection Watcher:127.0.0.1:37975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 233537 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 233538 T2098 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 233539 T2128 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 233540 T2098 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 233540 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233541 T2128 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 233541 T2128 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C38 name=collection1 org.apache.solr.core.SolrCore@164bbee7 url=http://127.0.0.1:13902/a/xe/collection1 node=127.0.0.1:13902_a%2Fxe C38_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:13902/a/xe, core=collection1, node_name=127.0.0.1:13902_a%2Fxe}
[junit4] 2> 233541 T2128 C38 P13902 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:13902/a/xe/collection1/
[junit4] 2> 233541 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 233542 T2128 C38 P13902 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 233542 T2128 C38 P13902 oasc.SyncStrategy.syncToMe http://127.0.0.1:13902/a/xe/collection1/ has no replicas
[junit4] 2> 233542 T2128 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:13902/a/xe/collection1/ shard1
[junit4] 2> 233543 T2128 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 233544 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233544 T2124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 233544 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 233546 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233547 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 233549 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233650 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 233650 T2124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 233699 T2128 oasc.ZkController.register We are http://127.0.0.1:13902/a/xe/collection1/ and leader is http://127.0.0.1:13902/a/xe/collection1/
[junit4] 2> 233700 T2128 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:13902/a/xe
[junit4] 2> 233700 T2128 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 233700 T2128 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 233700 T2128 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 233702 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233702 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233702 T2128 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 233702 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233703 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 233703 T2121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:13902/a/xe",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:13902_a%2Fxe",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 233705 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 233807 T2133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 233807 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 234645 T2098 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 234645 T2098 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 234648 T2098 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:33669
[junit4] 2> 234648 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 234649 T2098 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 234649 T2098 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003
[junit4] 2> 234649 T2098 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/'
[junit4] 2> 234679 T2098 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/solr.xml
[junit4] 2> 234733 T2098 oasc.CoreContainer.<init> New CoreContainer 322531860
[junit4] 2> 234734 T2098 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/]
[junit4] 2> 234735 T2098 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 234735 T2098 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 234735 T2098 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 234735 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 234735 T2098 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 234736 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 234736 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 234736 T2098 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 234736 T2098 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 234737 T2098 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 234737 T2098 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 234737 T2098 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 234738 T2098 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 234738 T2098 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37975/solr
[junit4] 2> 234738 T2098 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 234739 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 234740 T2144 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@207088e8 name:ZooKeeperConnection Watcher:127.0.0.1:37975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 234741 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 234743 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 234744 T2146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20d7aa52 name:ZooKeeperConnection Watcher:127.0.0.1:37975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 234744 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 234748 T2098 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 235751 T2098 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33669_a%2Fxe
[junit4] 2> 235752 T2098 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33669_a%2Fxe
[junit4] 2> 235754 T2133 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 235754 T2147 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 235754 T2131 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 235761 T2148 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 235761 T2148 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 235762 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 235762 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 235762 T2148 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 235762 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 235763 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 235764 T2121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33669/a/xe",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33669_a%2Fxe",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 235764 T2121 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 235764 T2121 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 235766 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 235766 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 235766 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 235766 T2132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 236763 T2148 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 236764 T2148 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 236764 T2148 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 236764 T2148 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 236765 T2148 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 236765 T2148 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/'
[junit4] 2> 236767 T2148 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 236768 T2148 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 236769 T2148 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 236835 T2148 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 236898 T2148 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 236899 T2148 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 236915 T2148 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 237371 T2148 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 237372 T2148 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 237372 T2148 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 237387 T2148 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 237390 T2148 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 237410 T2148 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 237415 T2148 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 237420 T2148 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 237421 T2148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 237421 T2148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 237421 T2148 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 237422 T2148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 237422 T2148 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 237422 T2148 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 237423 T2148 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 237423 T2148 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/data/
[junit4] 2> 237423 T2148 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f8c26ff
[junit4] 2> 237425 T2148 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/data
[junit4] 2> 237426 T2148 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/data/index/
[junit4] 2> 237427 T2148 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 237427 T2148 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-003/collection1/data/index
[junit4] 2> 237427 T2148 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=1.2470703125, floorSegmentMB=2.1884765625, forceMergeDeletesPctAllowed=23.277766949976346, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 237428 T2148 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@659d4591 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@347c2827),segFN=segments_1,generation=1}
[junit4] 2> 237428 T2148 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 237436 T2148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 237436 T2148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 237437 T2148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 237437 T2148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 237437 T2148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 237437 T2148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 237438 T2148 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 237438 T2148 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 237438 T2148 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 237439 T2148 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 237440 T2148 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 237440 T2148 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 237441 T2148 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 237442 T2148 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 237442 T2148 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 237443 T2148 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 237443 T2148 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 237444 T2148 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 237444 T2148 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 237444 T2148 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 237445 T2148 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 237464 T2148 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 237465 T2148 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 237467 T2148 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 237469 T2148 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 237471 T2148 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 237477 T2148 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 237477 T2148 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 237478 T2148 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=90.3037109375, floorSegmentMB=0.8642578125, forceMergeDeletesPctAllowed=12.1405079704558, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 237478 T2148 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@659d4591 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@347c2827),segFN=segments_1,generation=1}
[junit4] 2> 237479 T2148 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 237480 T2148 oass.SolrIndexSearcher.<init> Opening Searcher@15ed71f4[collection1] main
[junit4] 2> 237481 T2148 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 237482 T2148 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 237482 T2148 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 237482 T2148 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 237482 T2148 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 237483 T2148 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 237483 T2148 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 237484 T2148 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 237484 T2148 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 237484 T2148 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 237489 T2149 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15ed71f4[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 237494 T2148 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 237495 T2152 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33669/a/xe collection:collection1 shard:shard1
[junit4] 2> 237495 T2098 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 237496 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 237496 T2152 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 237500 T2152 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 237501 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237502 T2152 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 237502 T2152 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C39 name=collection1 org.apache.solr.core.SolrCore@17535669 url=http://127.0.0.1:33669/a/xe/collection1 node=127.0.0.1:33669_a%2Fxe C39_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:33669/a/xe, core=collection1, node_name=127.0.0.1:33669_a%2Fxe}
[junit4] 2> 237502 T2152 C39 P33669 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33669/a/xe/collection1/
[junit4] 2> 237502 T2152 C39 P33669 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 237502 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 237503 T2152 C39 P33669 oasc.SyncStrategy.syncToMe http://127.0.0.1:33669/a/xe/collection1/ has no replicas
[junit4] 2> 237503 T2152 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33669/a/xe/collection1/ shard1
[junit4] 2> 237503 T2152 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 237504 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237505 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237505 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237505 T2132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237507 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237508 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 237510 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237611 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237611 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237611 T2132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237660 T2152 oasc.ZkController.register We are http://127.0.0.1:33669/a/xe/collection1/ and leader is http://127.0.0.1:33669/a/xe/collection1/
[junit4] 2> 237661 T2152 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33669/a/xe
[junit4] 2> 237661 T2152 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 237661 T2152 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 237661 T2152 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 237663 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237663 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237663 T2152 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 237663 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237664 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 237664 T2121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:33669/a/xe",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:33669_a%2Fxe",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 237666 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 237767 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237767 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 237767 T2133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 238589 T2098 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 238590 T2098 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 238592 T2098 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25102
[junit4] 2> 238592 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 238593 T2098 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 238593 T2098 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004
[junit4] 2> 238593 T2098 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/'
[junit4] 2> 238623 T2098 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/solr.xml
[junit4] 2> 238677 T2098 oasc.CoreContainer.<init> New CoreContainer 340173289
[junit4] 2> 238677 T2098 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/]
[junit4] 2> 238679 T2098 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 238679 T2098 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 238679 T2098 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 238679 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 238679 T2098 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 238680 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 238680 T2098 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 238680 T2098 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 238680 T2098 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 238681 T2098 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 238681 T2098 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 238681 T2098 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 238682 T2098 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 238682 T2098 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:37975/solr
[junit4] 2> 238682 T2098 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 238683 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 238684 T2163 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5188dc4c name:ZooKeeperConnection Watcher:127.0.0.1:37975 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 238685 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 238687 T2098 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 238688 T2165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52ca77d4 name:ZooKeeperConnection Watcher:127.0.0.1:37975/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 238688 T2098 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 238692 T2098 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 239695 T2098 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25102_a%2Fxe
[junit4] 2> 239696 T2098 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25102_a%2Fxe
[junit4] 2> 239698 T2133 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 239698 T2147 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 239698 T2166 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 239698 T2131 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 239718 T2167 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 239719 T2167 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 239720 T2167 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 239720 T2133 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 239720 T2124 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 239720 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 239721 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 239722 T2121 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:25102/a/xe",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:25102_a%2Fxe",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 239722 T2121 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 239722 T2121 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 239724 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 239724 T2166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 239724 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 239724 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 239724 T2132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 240720 T2167 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 240721 T2167 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 240721 T2167 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 240721 T2167 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 240722 T2167 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 240722 T2167 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/'
[junit4] 2> 240725 T2167 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 240725 T2167 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 240726 T2167 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 240792 T2167 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 240842 T2167 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 240843 T2167 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 240858 T2167 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 241298 T2167 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 241299 T2167 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 241300 T2167 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 241309 T2167 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 241311 T2167 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 241326 T2167 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 241330 T2167 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 241332 T2167 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 241333 T2167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 241333 T2167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 241333 T2167 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 241334 T2167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 241334 T2167 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 241334 T2167 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 241334 T2167 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 241334 T2167 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/data/
[junit4] 2> 241335 T2167 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3f8c26ff
[junit4] 2> 241336 T2167 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/data
[junit4] 2> 241337 T2167 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/data/index/
[junit4] 2> 241337 T2167 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 241338 T2167 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-AB9A8892F2A68BB5-001/tempDir-004/collection1/data/index
[junit4] 2> 241338 T2167 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=1.2470703125, floorSegmentMB=2.1884765625, forceMergeDeletesPctAllowed=23.277766949976346, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 241338 T2167 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@65719564 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64c9932f),segFN=segments_1,generation=1}
[junit4] 2> 241339 T2167 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 241343 T2167 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 241344 T2167 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 241344 T2167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 241344 T2167 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 241344 T2167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 241344 T2167 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 241345 T2167 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 241345 T2167 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 241345 T2167 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 241346 T2167 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 241346 T2167 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 241347 T2167 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 241347 T2167 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 241347 T2167 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 241348 T2167 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 241348 T2167 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 241348 T2167 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 241349 T2167 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 241349 T2167 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 241349 T2167 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 241350 T2167 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 241362 T2167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 241363 T2167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 241365 T2167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 241366 T2167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 241367 T2167 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 241371 T2167 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 241371 T2167 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 241372 T2167 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=8, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=90.3037109375, floorSegmentMB=0.8642578125, forceMergeDeletesPctAllowed=12.1405079704558, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 241373 T2167 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=BaseDirectoryWrapper(RAMDirectory@65719564 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@64c9932f),segFN=segments_1,generation=1}
[junit4] 2> 241373 T2167 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 241373 T2167 oass.SolrIndexSearcher.<init> Opening Searcher@19220923[collection1] main
[junit4] 2> 241374 T2167 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 241374 T2167 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 241374 T2167 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 241375 T2167 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 241375 T2167 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 241375 T2167 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 241375 T2167 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 241376 T2167 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 241376 T2167 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 241376 T2167 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 241379 T2168 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@19220923[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 241382 T2167 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 241383 T2171 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25102/a/xe collection:collection1 shard:shard2
[junit4] 2> 241383 T2098 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 241384 T2098 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 241384 T2171 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 241387 T2171 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 241388 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 241388 T2171 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 241389 T2171 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C40 name=collection1 org.apache.solr.core.SolrCore@189faa75 url=http://127.0.0.1:25102/a/xe/collection1 node=127.0.0.1:25102_a%2Fxe C40_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:25102/a/xe, core=collection1, node_name=127.0.0.1:25102_a%2Fxe}
[junit4] 2> 241389 T2171 C40 P25102 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25102/a/xe/collection1/
[junit4] 2> 241389 T2171 C40 P25102 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 241389 T2121 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 241389 T2171 C40 P25102 oasc.SyncStrategy.syncToMe http://127.0.0.1:25102/a/xe/collection1/ has no replicas
[junit4] 2> 241389 T2171 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25102/a/xe/collection1/ shard2
[junit4] 2> 241389 T2171 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 241391 T2132 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 241391 T2166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 241391 T2147 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 241391 T2132 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 241391 T2131 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged pa
[...truncated too long message...]
.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=780101, name=qtp66094268-780101, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=780096, name=qtp860896798-780096, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #11: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=780155, name=RecoveryThread, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1360)
[junit4] > at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.httpUriRequest(HttpSolrServer.java:241)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.httpUriRequest(HttpSolrServer.java:230)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:610)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:371)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:235)Throwable #12: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=779767, name=qtp66094268-779767, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > Caused by: java.lang.OutOfMemoryError: unable to create new native thread
[junit4] > at __randomizedtesting.SeedInfo.seed([AB9A8892F2A68BB5]:0)
[junit4] > at java.lang.Thread.start0(Native Method)
[junit4] > at java.lang.Thread.start(Thread.java:714)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1047)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
[junit4] > at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:665)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3570756 T771884 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=494), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=494)), sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=zh_SG, timezone=Africa/Harare
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=193193280,total=512229376
[junit4] 2> NOTE: All tests run in this JVM: [ChaosMonkeySafeLeaderTest, TestLazyCores, TestCollectionAPI, SampleTest, TimeZoneUtilsTest, DistributedQueryElevationComponentTest, DistribCursorPagingTest, TestGroupingSearch, ShardSplitTest, DocValuesMissingTest, DOMUtilTest, TestNoOpRegenerator, TestPseudoReturnFields, TestMissingGroups, DeleteInactiveReplicaTest, TestLeaderElectionZkExpiry, ScriptEngineTest, CopyFieldTest, SpellingQueryConverterTest, SolrRequestParserTest, DistributedQueryComponentCustomSortTest, LukeRequestHandlerTest, TestSolrQueryParserDefaultOperatorResource, TestElisionMultitermQuery, TestQuerySenderNoQuery, TestQueryUtils, TestIndexingPerformance, QueryElevationComponentTest, CoreAdminRequestStatusTest, TestCSVResponseWriter, TestAddFieldRealTimeGet, TestPHPSerializedResponseWriter, FileUtilsTest, SpatialFilterTest, TestCoreContainer, TestBadConfig, AlternateDirectoryTest, TestImplicitCoreProperties, TestDocumentBuilder, SearchHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, DeleteReplicaTest, CurrencyFieldXmlFileTest, AddBlockUpdateTest, SolrCoreTest, DistributedSuggestComponentTest, AutoCommitTest, OpenCloseCoreStressTest, TestPhraseSuggestions, TestMultiCoreConfBootstrap, TestCharFilters, TestSerializedLuceneMatchVersion, TestBinaryField, TestRTGBase, TestFiltering, TestSuggestSpellingConverter, BasicZkTest, TestIndexSearcher, SystemInfoHandlerTest, FileBasedSpellCheckerTest, TestUniqueKeyFieldResource, TestLMDirichletSimilarityFactory, TestDocBasedVersionConstraints, SimplePostToolTest, TestOmitPositions, CoreMergeIndexesAdminHandlerTest, TestRandomFaceting, HighlighterMaxOffsetTest, TestRecovery, SortByFunctionTest, AddSchemaFieldsUpdateProcessorFactoryTest, SolrInfoMBeanTest, TestManagedSchemaFieldResource, TestIBSimilarityFactory, HdfsLockFactoryTest, TestSolrQueryParser, FullSolrCloudDistribCmdsTest, DistributedQueryComponentOptimizationTest, TestCursorMarkWithoutUniqueKey, ConnectionManagerTest, ResponseHeaderTest, TestSchemaNameResource, NumericFieldsTest, TestInfoStreamLogging, SOLR749Test, LoggingHandlerTest, CSVRequestHandlerTest, BasicDistributedZk2Test, TestExtendedDismaxParser, TestManagedStopFilterFactory, DistanceFunctionTest, TestSchemaSimilarityResource, TestJoin, CacheHeaderTest, TestShardHandlerFactory, TestMergePolicyConfig, RequestHandlersTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest]
[junit4] Completed on J1 in 378.59s, 1 test, 1 error <<< FAILURES!
[...truncated 685 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 411 suites, 1707 tests, 1 error, 1 failure, 50 ignored (8 assumptions)
Total time: 261 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 460166856 bytes
Compression is 0.0%
Took 1 min 14 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 591 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/591/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([16B88401E7F4DB3E:975E0A1990ABBB02]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12474 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/init-core-data-001
[junit4] 2> 3690398 T17981 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 3690398 T17981 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_fn/uc
[junit4] 2> 3690401 T17981 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 3690401 T17981 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3690402 T17982 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 3690502 T17981 oasc.ZkTestServer.run start zk server on port:12730
[junit4] 2> 3690503 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3690506 T17988 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7aa6d46e name:ZooKeeperConnection Watcher:127.0.0.1:12730 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3690506 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3690506 T17981 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 3690509 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3690510 T17990 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c0d3f0f name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3690510 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3690511 T17981 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 3690512 T17981 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 3690514 T17981 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 3690515 T17981 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 3690517 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3690517 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 3690520 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 3690520 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 3690523 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3690523 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3690525 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3690526 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 3690527 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3690528 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 3690530 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3690530 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 3690532 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3690533 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 3690535 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3690535 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 3690537 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3690538 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3690539 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3690540 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 3690542 T17981 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3690542 T17981 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 3690545 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3690546 T17992 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64ae7156 name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3690546 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3691575 T17981 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3691578 T17981 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3691582 T17981 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:36468
[junit4] 2> 3691585 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3691585 T17981 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3691585 T17981 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002
[junit4] 2> 3691585 T17981 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/'
[junit4] 2> 3691618 T17981 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/solr.xml
[junit4] 2> 3691670 T17981 oasc.CoreContainer.<init> New CoreContainer 1441868522
[junit4] 2> 3691670 T17981 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/]
[junit4] 2> 3691671 T17981 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3691671 T17981 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3691671 T17981 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3691672 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3691672 T17981 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3691672 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3691672 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3691673 T17981 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3691673 T17981 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3691675 T17981 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3691676 T17981 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3691676 T17981 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3691677 T17981 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3691677 T17981 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12730/solr
[junit4] 2> 3691677 T17981 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3691678 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3691679 T18003 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16b89129 name:ZooKeeperConnection Watcher:127.0.0.1:12730 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3691680 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3691682 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3691683 T18005 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3754212f name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3691683 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3691684 T17981 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 3691686 T17981 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 3691688 T17981 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 3691690 T17981 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 3691692 T17981 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 3691695 T17981 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 3691696 T17981 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36468__fn%2Fuc
[junit4] 2> 3691697 T17981 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36468__fn%2Fuc
[junit4] 2> 3691699 T17981 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 3691700 T17981 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 3691703 T17981 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36468__fn%2Fuc
[junit4] 2> 3691703 T17981 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 3691705 T17981 oasc.Overseer.start Overseer (id=92223578863894532-127.0.0.1:36468__fn%2Fuc-n_0000000000) starting
[junit4] 2> 3691707 T17981 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 3691713 T18007 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 3691713 T17981 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 3691715 T17981 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 3691716 T17981 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3691718 T18006 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 3691723 T18008 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 3691724 T18008 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3691724 T18008 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3691724 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3691726 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3691726 T18006 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36468/_fn/uc",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:36468__fn%2Fuc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3691726 T18006 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 3691726 T18006 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3691728 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3691728 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3692725 T18008 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3692726 T18008 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 3692727 T18008 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3692727 T18008 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3692728 T18008 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3692728 T18008 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/collection1/'
[junit4] 2> 3692730 T18008 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 3692731 T18008 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 3692732 T18008 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 3692794 T18008 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3692856 T18008 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3692857 T18008 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3692868 T18008 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3693279 T18008 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3693279 T18008 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3693280 T18008 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3693292 T18008 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3693295 T18008 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3693306 T18008 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3693310 T18008 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3693314 T18008 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3693315 T18008 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3693315 T18008 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3693315 T18008 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3693316 T18008 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3693316 T18008 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3693316 T18008 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 3693317 T18008 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3693317 T18008 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-002/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-001/control/data/
[junit4] 2> 3693317 T18008 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c7a92e6
[junit4] 2> 3693318 T18008 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-001/control/data
[junit4] 2> 3693318 T18008 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-001/control/data/index/
[junit4] 2> 3693318 T18008 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 3693319 T18008 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-001/control/data/index
[junit4] 2> 3693319 T18008 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3693320 T18008 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@523999e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a3771fa)),segFN=segments_1,generation=1}
[junit4] 2> 3693320 T18008 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3693324 T18008 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3693324 T18008 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3693325 T18008 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3693325 T18008 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3693325 T18008 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3693325 T18008 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3693326 T18008 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3693326 T18008 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3693327 T18008 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3693327 T18008 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3693328 T18008 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3693328 T18008 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3693328 T18008 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3693329 T18008 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3693329 T18008 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3693330 T18008 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3693330 T18008 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3693330 T18008 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3693331 T18008 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3693331 T18008 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3693331 T18008 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3693350 T18008 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3693352 T18008 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3693354 T18008 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3693356 T18008 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3693358 T18008 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3693363 T18008 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3693363 T18008 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3693364 T18008 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=57.248046875, floorSegmentMB=1.1875, forceMergeDeletesPctAllowed=8.039933390241579, segmentsPerTier=27.0, maxCFSSegmentSizeMB=0.7099609375, noCFSRatio=0.0
[junit4] 2> 3693365 T18008 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@523999e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5a3771fa)),segFN=segments_1,generation=1}
[junit4] 2> 3693365 T18008 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3693365 T18008 oass.SolrIndexSearcher.<init> Opening Searcher@59f2059b[collection1] main
[junit4] 2> 3693365 T18008 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 3693366 T18008 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3693366 T18008 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3693367 T18008 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3693367 T18008 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3693367 T18008 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3693368 T18008 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 3693368 T18008 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3693369 T18008 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 3693370 T18008 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 3693371 T18008 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3693371 T18008 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3693374 T18010 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@59f2059b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3693376 T18008 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3693377 T18013 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:36468/_fn/uc collection:control_collection shard:shard1
[junit4] 2> 3693377 T17981 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 3693378 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3693378 T18013 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 3693382 T18013 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3693383 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3693383 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693384 T18015 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f17be61 name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3693384 T18013 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3693385 T18013 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 3693385 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3693384 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> ASYNC NEW_CORE C8814 name=collection1 org.apache.solr.core.SolrCore@7dc3f444 url=https://127.0.0.1:36468/_fn/uc/collection1 node=127.0.0.1:36468__fn%2Fuc C8814_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:36468/_fn/uc, core=collection1, node_name=127.0.0.1:36468__fn%2Fuc}
[junit4] 2> 3693385 T18013 C8814 P36468 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:36468/_fn/uc/collection1/
[junit4] 2> 3693386 T18013 C8814 P36468 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3693386 T18013 C8814 P36468 oasc.SyncStrategy.syncToMe https://127.0.0.1:36468/_fn/uc/collection1/ has no replicas
[junit4] 2> 3693386 T17981 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3693386 T18013 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:36468/_fn/uc/collection1/ shard1
[junit4] 2> 3693387 T18013 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 3693387 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693387 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3693388 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 3693389 T17981 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3693391 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693392 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3693393 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693495 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3693495 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3693543 T18013 oasc.ZkController.register We are https://127.0.0.1:36468/_fn/uc/collection1/ and leader is https://127.0.0.1:36468/_fn/uc/collection1/
[junit4] 2> 3693544 T18013 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:36468/_fn/uc
[junit4] 2> 3693544 T18013 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3693544 T18013 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 3693544 T18013 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3693545 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693545 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693545 T18013 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3693545 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693547 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3693547 T18006 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:36468/_fn/uc",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:36468__fn%2Fuc",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3693549 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3693650 T18018 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3693650 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3694700 T17981 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 3694700 T17981 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3694705 T17981 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3694706 T17981 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:38377
[junit4] 2> 3694708 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3694708 T17981 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3694708 T17981 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003
[junit4] 2> 3694709 T17981 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/'
[junit4] 2> 3694752 T17981 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/solr.xml
[junit4] 2> 3694812 T17981 oasc.CoreContainer.<init> New CoreContainer 980953951
[junit4] 2> 3694812 T17981 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/]
[junit4] 2> 3694813 T17981 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3694814 T17981 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3694814 T17981 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3694814 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3694814 T17981 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3694815 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3694815 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3694815 T17981 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3694815 T17981 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3694817 T17981 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3694819 T17981 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3694819 T17981 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3694819 T17981 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3694820 T17981 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12730/solr
[junit4] 2> 3694820 T17981 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3694821 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3694822 T18029 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@405c7c1c name:ZooKeeperConnection Watcher:127.0.0.1:12730 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3694822 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3694823 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3694824 T18031 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34dd6da7 name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3694824 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3694828 T17981 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3695830 T17981 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38377__fn%2Fuc
[junit4] 2> 3695831 T17981 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38377__fn%2Fuc
[junit4] 2> 3695832 T18018 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3695833 T18032 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3695832 T18016 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 3695838 T18033 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3695838 T18033 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3695839 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3695839 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3695839 T18033 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3695839 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3695840 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3695841 T18006 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38377/_fn/uc",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38377__fn%2Fuc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3695841 T18006 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 3695841 T18006 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 3695843 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3695843 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3695843 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3695843 T18017 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3696840 T18033 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3696841 T18033 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3696842 T18033 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3696842 T18033 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3696843 T18033 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3696843 T18033 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/'
[junit4] 2> 3696845 T18033 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 3696845 T18033 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 3696846 T18033 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 3696920 T18033 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3696971 T18033 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3696972 T18033 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3696983 T18033 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3697329 T18033 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3697330 T18033 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3697330 T18033 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3697339 T18033 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3697341 T18033 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3697350 T18033 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3697353 T18033 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3697356 T18033 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3697356 T18033 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3697356 T18033 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3697357 T18033 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3697357 T18033 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3697357 T18033 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3697358 T18033 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3697358 T18033 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3697358 T18033 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/data/
[junit4] 2> 3697358 T18033 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c7a92e6
[junit4] 2> 3697359 T18033 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/data
[junit4] 2> 3697359 T18033 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/data/index/
[junit4] 2> 3697359 T18033 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3697359 T18033 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-003/collection1/data/index
[junit4] 2> 3697360 T18033 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3697360 T18033 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@5c528c6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5f47fb8d)),segFN=segments_1,generation=1}
[junit4] 2> 3697360 T18033 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3697364 T18033 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3697364 T18033 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3697364 T18033 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3697364 T18033 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3697364 T18033 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3697365 T18033 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3697365 T18033 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3697365 T18033 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3697365 T18033 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3697366 T18033 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3697366 T18033 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3697367 T18033 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3697367 T18033 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3697367 T18033 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3697368 T18033 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3697368 T18033 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3697368 T18033 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3697368 T18033 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3697369 T18033 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3697369 T18033 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3697369 T18033 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3697383 T18033 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3697384 T18033 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3697386 T18033 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3697387 T18033 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3697388 T18033 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3697392 T18033 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3697392 T18033 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3697392 T18033 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=57.248046875, floorSegmentMB=1.1875, forceMergeDeletesPctAllowed=8.039933390241579, segmentsPerTier=27.0, maxCFSSegmentSizeMB=0.7099609375, noCFSRatio=0.0
[junit4] 2> 3697393 T18033 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@5c528c6c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5f47fb8d)),segFN=segments_1,generation=1}
[junit4] 2> 3697393 T18033 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3697393 T18033 oass.SolrIndexSearcher.<init> Opening Searcher@3feb7b02[collection1] main
[junit4] 2> 3697393 T18033 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3697394 T18033 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3697394 T18033 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3697395 T18033 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3697395 T18033 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3697395 T18033 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3697396 T18033 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3697396 T18033 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3697396 T18033 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3697396 T18033 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3697399 T18034 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3feb7b02[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3697401 T18033 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3697402 T18037 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:38377/_fn/uc collection:collection1 shard:shard2
[junit4] 2> 3697402 T17981 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 3697403 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3697403 T18037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 3697407 T18037 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 3697408 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697409 T18037 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3697409 T18037 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C8815 name=collection1 org.apache.solr.core.SolrCore@6e7bc72b url=https://127.0.0.1:38377/_fn/uc/collection1 node=127.0.0.1:38377__fn%2Fuc C8815_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:38377/_fn/uc, core=collection1, node_name=127.0.0.1:38377__fn%2Fuc}
[junit4] 2> 3697409 T18037 C8815 P38377 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:38377/_fn/uc/collection1/
[junit4] 2> 3697409 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3697409 T18037 C8815 P38377 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3697410 T18037 C8815 P38377 oasc.SyncStrategy.syncToMe https://127.0.0.1:38377/_fn/uc/collection1/ has no replicas
[junit4] 2> 3697410 T18037 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:38377/_fn/uc/collection1/ shard2
[junit4] 2> 3697410 T18037 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 3697411 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697412 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697412 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697412 T18017 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697414 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697415 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3697416 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697518 T18017 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697518 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697518 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697567 T18037 oasc.ZkController.register We are https://127.0.0.1:38377/_fn/uc/collection1/ and leader is https://127.0.0.1:38377/_fn/uc/collection1/
[junit4] 2> 3697568 T18037 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:38377/_fn/uc
[junit4] 2> 3697568 T18037 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3697568 T18037 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3697568 T18037 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3697569 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697569 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697569 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697569 T18037 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3697570 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3697571 T18006 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"https://127.0.0.1:38377/_fn/uc",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:38377__fn%2Fuc",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 3697572 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3697674 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697674 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3697674 T18018 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 3698795 T17981 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 3698795 T17981 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 3698799 T17981 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 3698800 T17981 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:21658
[junit4] 2> 3698803 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 3698803 T17981 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3698803 T17981 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004
[junit4] 2> 3698803 T17981 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/'
[junit4] 2> 3698839 T17981 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/solr.xml
[junit4] 2> 3698909 T17981 oasc.CoreContainer.<init> New CoreContainer 1971327179
[junit4] 2> 3698910 T17981 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/]
[junit4] 2> 3698911 T17981 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 3698911 T17981 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 3698911 T17981 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 3698911 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3698912 T17981 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3698912 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3698912 T17981 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3698912 T17981 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3698913 T17981 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3698914 T17981 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 3698916 T17981 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3698916 T17981 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3698916 T17981 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 3698917 T17981 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:12730/solr
[junit4] 2> 3698917 T17981 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3698918 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3698919 T18048 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@187e507d name:ZooKeeperConnection Watcher:127.0.0.1:12730 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3698919 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3698921 T17981 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3698922 T18050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c99f2ff name:ZooKeeperConnection Watcher:127.0.0.1:12730/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3698922 T17981 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3698927 T17981 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3699930 T17981 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21658__fn%2Fuc
[junit4] 2> 3699931 T17981 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21658__fn%2Fuc
[junit4] 2> 3699933 T18016 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3699933 T18032 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3699933 T18018 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3699933 T18051 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 3699939 T18052 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3699939 T18052 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3699940 T18018 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3699940 T18017 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3699940 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3699940 T18052 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 3699941 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3699942 T18006 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:21658/_fn/uc",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:21658__fn%2Fuc",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 3699942 T18006 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 3699942 T18006 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 3699943 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3699944 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3699944 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3699944 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3699944 T18051 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3700941 T18052 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3700942 T18052 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3700943 T18052 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3700943 T18052 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3700943 T18052 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3700944 T18052 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/'
[junit4] 2> 3700945 T18052 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 3700946 T18052 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 3700946 T18052 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 3701013 T18052 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 3701060 T18052 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3701061 T18052 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3701072 T18052 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3701517 T18052 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3701518 T18052 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 3701518 T18052 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3701535 T18052 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3701538 T18052 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3701550 T18052 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3701555 T18052 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3701559 T18052 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3701560 T18052 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3701560 T18052 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3701560 T18052 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3701561 T18052 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3701561 T18052 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3701562 T18052 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3701562 T18052 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3701562 T18052 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/data/
[junit4] 2> 3701562 T18052 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c7a92e6
[junit4] 2> 3701563 T18052 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/data
[junit4] 2> 3701563 T18052 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/data/index/
[junit4] 2> 3701564 T18052 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 3701564 T18052 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-004/collection1/data/index
[junit4] 2> 3701564 T18052 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 3701565 T18052 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@92c6d27 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4dce35d3)),segFN=segments_1,generation=1}
[junit4] 2> 3701566 T18052 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3701570 T18052 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 3701570 T18052 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3701571 T18052 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3701571 T18052 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3701571 T18052 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3701571 T18052 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3701572 T18052 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3701572 T18052 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3701573 T18052 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3701573 T18052 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3701574 T18052 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3701574 T18052 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3701575 T18052 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3701575 T18052 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3701576 T18052 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3701576 T18052 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3701576 T18052 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3701577 T18052 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3701577 T18052 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3701578 T18052 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3701578 T18052 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3701609 T18052 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3701611 T18052 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3701613 T18052 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3701615 T18052 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3701618 T18052 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3701624 T18052 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3701624 T18052 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3701625 T18052 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=57.248046875, floorSegmentMB=1.1875, forceMergeDeletesPctAllowed=8.039933390241579, segmentsPerTier=27.0, maxCFSSegmentSizeMB=0.7099609375, noCFSRatio=0.0
[junit4] 2> 3701625 T18052 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@92c6d27 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4dce35d3)),segFN=segments_1,generation=1}
[junit4] 2> 3701625 T18052 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3701626 T18052 oass.SolrIndexSearcher.<init> Opening Searcher@3376e486[collection1] main
[junit4] 2> 3701626 T18052 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3701627 T18052 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3701627 T18052 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 3701628 T18052 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3701628 T18052 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 3701628 T18052 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3701629 T18052 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 3701629 T18052 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3701629 T18052 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 3701630 T18052 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3701633 T18053 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3376e486[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 3701636 T18052 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3701637 T18056 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:21658/_fn/uc collection:collection1 shard:shard1
[junit4] 2> 3701637 T17981 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 3701638 T18056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 3701638 T17981 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 3701641 T18056 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3701643 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3701643 T18056 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3701643 T18056 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C8816 name=collection1 org.apache.solr.core.SolrCore@7c1ebe65 url=https://127.0.0.1:21658/_fn/uc/collection1 node=127.0.0.1:21658__fn%2Fuc C8816_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:21658/_fn/uc, core=collection1, node_name=127.0.0.1:21658__fn%2Fuc}
[junit4] 2> 3701643 T18056 C8816 P21658 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:21658/_fn/uc/collection1/
[junit4] 2> 3701644 T18056 C8816 P21658 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 3701644 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3701644 T18056 C8816 P21658 oasc.SyncStrategy.syncToMe https://127.0.0.1:21658/_fn/uc/collection1/ has no replicas
[junit4] 2> 3701644 T18056 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:21658/_fn/uc/collection1/ shard1
[junit4] 2> 3701645 T18056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 3701646 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3701647 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701647 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701647 T18051 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701647 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701649 T18006 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 3701651 T18009 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3701753 T18051 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701753 T18009 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701753 T18016 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701753 T18032 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 3701801 T18056 oasc.ZkController.register We are https://127.0.0.1:21658/_fn/uc/collection1/ and leader is https://127.0.0.1:21658/_fn/uc/collection1/
[junit4] 2> 3701802 T18056 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:21658/_fn/uc
[junit4] 2> 3701802 T18056 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 3701802 T18056 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3701802 T18056 oasc.ZkController.publish numShards not fou
[...truncated too long message...]
rdsplit_shard3_replica1/data/index
[junit4] 2> 3738821 T17981 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data;done=false>>]
[junit4] 2> 3738822 T17981 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001/tempDir-006/ocptest_shardsplit_shard3_replica1/data
[junit4] 2> 3738822 T19559 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:43541__fn%2Fuc
[junit4] 2> 3738824 T19554 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 3738824 T19561 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 3738825 T17981 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/_fn/uc,null}
[junit4] 2> 3743723 T17981 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 3743725 T17981 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:12730 12730
[junit4] 2> 3743796 T17982 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:12730 12730
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=16B88401E7F4DB3E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=pt_BR -Dtests.timezone=ROK -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 53.4s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([16B88401E7F4DB3E:975E0A1990ABBB02]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3743818 T17981 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-16B88401E7F4DB3E-001
[junit4] 2> 53424 T17980 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=pt_BR, timezone=ROK
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=349712448,total=529530880
[junit4] 2> NOTE: All tests run in this JVM: [TestDFRSimilarityFactory, TestCloudManagedSchemaConcurrent, TestChildDocTransformer, TestHighlightDedupGrouping, HdfsBasicDistributedZk2Test, CollectionsAPIAsyncDistributedZkTest, TestCustomSort, TestShortCircuitedRequests, XsltUpdateRequestHandlerTest, StressHdfsTest, ResponseLogComponentTest, DistributedSpellCheckComponentTest, DistribDocExpirationUpdateProcessorTest, OverseerTest, SolrTestCaseJ4Test, TestFileDictionaryLookup, PingRequestHandlerTest, TestReplicationHandler, TestCollapseQParserPlugin, TestUtils, StatelessScriptUpdateProcessorFactoryTest, TestReplicationHandlerBackup, TestLMJelinekMercerSimilarityFactory, EchoParamsTest, TestStressRecovery, TestIntervalFaceting, FastVectorHighlighterTest, ReturnFieldsTest, TestFieldTypeCollectionResource, ShardRoutingTest, ZkCLITest, IndexSchemaTest, ClusterStateUpdateTest, TestQuerySenderListener, TestSearcherReuse, SliceStateTest, AtomicUpdatesTest, IndexBasedSpellCheckerTest, DistributedIntervalFacetingTest, TestSolrDeletionPolicy2, CircularListTest, ShowFileRequestHandlerTest, RegexBoostProcessorTest, PrimUtilsTest, LeaderElectionTest, UUIDUpdateProcessorFallbackTest, AliasIntegrationTest, TestSolrDeletionPolicy1, SpellPossibilityIteratorTest, CoreAdminCreateDiscoverTest, HdfsCollectionsAPIDistributedZkTest, TestFieldTypeResource, TestCloudManagedSchema, TestBM25SimilarityFactory, TestDefaultSimilarityFactory, AssignTest, DirectSolrSpellCheckerTest, TestSchemaResource, ParsingFieldUpdateProcessorsTest, XmlUpdateRequestHandlerTest, SuggesterTest, TestSolr4Spatial, TestJmxMonitoredMap, RequiredFieldsTest, ResourceLoaderTest, DisMaxRequestHandlerTest, TestSimpleQParserPlugin, BadComponentTest, TestStressUserVersions, OverseerCollectionProcessorTest, TestManagedSchema, AnalyticsQueryTest, PreAnalyzedUpdateProcessorTest, TestSurroundQueryParser, HighlighterConfigTest, ClusterStateTest, TestFuzzyAnalyzedSuggestions, TestFastOutputStream, TestManagedResource, QueryResultKeyTest, HighlighterTest, NotRequiredUniqueKeyTest, SuggesterTSTTest, PluginInfoTest, TestSolrXml, MergeStrategyTest, TestSolrXmlPersistence, TestRandomMergePolicy, TestSort, TestHashPartitioner, PeerSyncTest, TestRandomDVFaceting, TestRemoteStreaming, DocumentBuilderTest, TestDistributedMissingSort, DocValuesMultiTest, SpellCheckComponentTest, TestReloadAndDeleteDocs, CursorPagingTest, OpenCloseCoreStressTest, TestFaceting, TestRecovery, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestMultiCoreConfBootstrap, TestReload, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, SimpleFacetsTest, SolrCoreTest, SolrCmdDistributorTest, DirectUpdateHandlerTest, TestIndexSearcher, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, TestTrie, SpatialFilterTest, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, QueryEqualityTest, LukeRequestHandlerTest, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, TestJmxIntegration, TestCSVResponseWriter, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, TestComponentsName, CoreContainerCoreInitFailuresTest, TestLFUCache, UpdateParamsTest, MultiTermTest, TestDocSet, TestSearchPerf, MinimalSchemaTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, TestCodecSupport, SynonymTokenizerTest, TestNumberUtils, TestIBSimilarityFactory, LegacyHTMLStripCharFilterTest, OpenExchangeRatesOrgProviderTest, DateMathParserTest, TestSolrXMLSerializer, DateFieldTest, RAMDirectoryFactoryTest, FileUtilsTest, TestRTGBase, TestTolerantSearch, ConnectionManagerTest, DeleteReplicaTest, DeleteShardTest, MigrateRouteKeyTest, MultiThreadedOCPTest]
[junit4] Completed on J0 in 53.70s, 1 test, 1 failure <<< FAILURES!
[...truncated 254 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 411 suites, 1707 tests, 1 failure, 50 ignored (8 assumptions)
Total time: 233 minutes 44 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 667921207 bytes
Compression is 0.0%
Took 1 min 53 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 590 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/590/
2 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([53BB0B78D4E89FC5:57B3848BC64D70E4]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:144)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:131)
... 48 more
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:457)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:454)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:411)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:136)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
... 49 more
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([53BB0B78D4E89FC5:D25D8560A3B7FFF9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11473 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.OverseerTest-53BB0B78D4E89FC5-001/init-core-data-001
[junit4] 2> 132304 T483 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 132304 T483 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 132304 T483 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 132309 T483 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 132322 T483 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 132331 T484 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 132531 T483 oasc.ZkTestServer.run start zk server on port:34561
[junit4] 2> 132866 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 132915 T485 oazs.NIOServerCnxn.doIO WARN Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4] 2> 134169 T490 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2338eb14 name:ZooKeeperConnection Watcher:127.0.0.1:34561/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 134169 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 134176 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 134179 T492 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71ba2933 name:ZooKeeperConnection Watcher:127.0.0.1:34561 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 134179 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 134264 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 134266 T494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b224c45 name:ZooKeeperConnection Watcher:127.0.0.1:34561 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 134267 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 134273 T483 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 134300 T483 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 134324 T483 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 134327 T483 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 134329 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 134420 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 134433 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 134435 T496 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f1cf161 name:ZooKeeperConnection Watcher:127.0.0.1:34561/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 134435 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 134443 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 134446 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 134476 T483 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:34561_solr
[junit4] 2> 134477 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 134480 T483 oasc.Overseer.start Overseer (id=92213278593843203-127.0.0.1:34561_solr-n_0000000000) starting
[junit4] 2> 134508 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 134520 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 134524 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 134528 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 134551 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 134560 T498 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 134575 T497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 134576 T497 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 134577 T497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 134641 T497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 134642 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 134645 T497 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 134648 T497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 134649 T497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 134653 T500 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 134756 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 134770 T483 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34561 34561
[junit4] 2> 134889 T484 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34561 34561
[junit4] 2> 134890 T483 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 2> 134891 T497 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:34561_solr
[junit4] 2> 134900 T483 oas.SolrTestCaseJ4.setUp ###Starting testDoubleAssignment
[junit4] 2> 134902 T483 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 134902 T502 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 134904 T501 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 135002 T483 oasc.ZkTestServer.run start zk server on port:34567
[junit4] 2> 135003 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135007 T508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@102d0abe name:ZooKeeperConnection Watcher:127.0.0.1:34567/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135007 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135008 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135009 T510 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@548dcd63 name:ZooKeeperConnection Watcher:127.0.0.1:34567 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135010 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135012 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135014 T512 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17373182 name:ZooKeeperConnection Watcher:127.0.0.1:34567 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135014 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135014 T483 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 135018 T483 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 135021 T483 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 135024 T483 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 135026 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 135035 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135037 T514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24ff9528 name:ZooKeeperConnection Watcher:127.0.0.1:34567/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135037 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135039 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 135042 T483 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 135046 T515 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135046 T516 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135046 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135048 T518 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40af72ba name:ZooKeeperConnection Watcher:127.0.0.1:34567/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135048 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135049 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 135052 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 135058 T483 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:34567_solr
[junit4] 2> 135058 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 135061 T483 oasc.Overseer.start Overseer (id=92213278738284548-127.0.0.1:34567_solr-n_0000000000) starting
[junit4] 2> 135063 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 135068 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 135072 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 135076 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 135081 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 135086 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 135093 T520 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 135095 T519 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 135098 T519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 135099 T519 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 135099 T519 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 135111 T519 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 135115 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135116 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 135116 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 135598 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 135628 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 135636 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135636 T483 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 135638 T519 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 135639 T516 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 135639 T515 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 135641 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135643 T523 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b702644 name:ZooKeeperConnection Watcher:127.0.0.1:34567/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135643 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135645 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 135648 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135648 T483 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 135652 T516 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135652 T524 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135654 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135654 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 135655 T519 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 135656 T519 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 135659 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135662 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 135668 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135672 T521 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 135775 T524 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 135775 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 135776 T483 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 135780 T516 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 135780 T524 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 135782 T483 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34567 34567
[junit4] 2> 135825 T502 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34567 34567
[junit4] 2> 135826 T483 oas.SolrTestCaseJ4.tearDown ###Ending testDoubleAssignment
[junit4] 2> 135826 T519 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:34567_solr
[junit4] 2> 135827 T525 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 135836 T483 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
[junit4] 2> 135837 T483 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 135837 T526 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 135937 T483 oasc.ZkTestServer.run start zk server on port:34574
[junit4] 2> 135938 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135950 T532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@65c9a3b8 name:ZooKeeperConnection Watcher:127.0.0.1:34574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135950 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135953 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135955 T534 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c78440 name:ZooKeeperConnection Watcher:127.0.0.1:34574 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135955 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135956 T483 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 135960 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135962 T536 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26ebec38 name:ZooKeeperConnection Watcher:127.0.0.1:34574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135962 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135962 T483 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 135965 T483 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 135968 T483 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 135970 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 135974 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135975 T538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3852eabd name:ZooKeeperConnection Watcher:127.0.0.1:34574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135976 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135977 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 135981 T483 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 135984 T539 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135984 T540 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 135985 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 135986 T542 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40beb5f0 name:ZooKeeperConnection Watcher:127.0.0.1:34574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 135987 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 135988 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 135991 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 135996 T483 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:34574_solr
[junit4] 2> 135997 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 136000 T483 oasc.Overseer.start Overseer (id=92213278799495172-127.0.0.1:34574_solr-n_0000000000) starting
[junit4] 2> 136002 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 136006 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 136010 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 136015 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 136019 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 136023 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 136030 T544 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 136032 T543 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 137034 T545 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 137036 T543 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 137037 T543 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 137037 T543 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 137038 T543 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 137041 T545 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 137042 T539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 137042 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 137534 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 137545 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 137552 T545 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 137553 T483 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 137554 T543 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 137557 T545 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 137557 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 137560 T543 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 137560 T543 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 137563 T545 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 137666 T539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 137666 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 138035 T544 oasc.OverseerCollectionProcessor.run WARN Overseer cannot talk to ZK
[junit4] 2> 138722 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 138731 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 138733 T547 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5633c113 name:ZooKeeperConnection Watcher:127.0.0.1:34574/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 138733 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 138733 T543 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:34574_solr
[junit4] 2> 138734 T548 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 138738 T483 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:34574_solr
[junit4] 2> 138738 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 138741 T483 oasc.Overseer.start Overseer (id=92213278799495173-127.0.0.1:34574_solr-n_0000000001) starting
[junit4] 2> 138753 T550 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 138755 T549 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 138757 T549 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 138758 T549 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 138759 T549 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 138762 T551 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 138763 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 138763 T539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 138817 T551 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 138817 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 138819 T549 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 138824 T483 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/92213278799495171-node1_core1-n_0000000002
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:148)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] 2> at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
[junit4] 2> at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 138827 T540 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 138828 T549 oasc.SolrException.log ERROR Problem cleaning up collection in zk:collection1:org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:184)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:181)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:181)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:665)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:660)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:660)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:660)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:1100)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:358)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:240)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 138829 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 138829 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 138831 T551 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 138834 T551 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 138835 T549 oasc.Overseer$ClusterStateUpdater.setShardLeader ERROR Could not mark shard leader for non existing collection:collection1
[junit4] 2> 138838 T551 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 138941 T539 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 138941 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 139833 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 139833 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 140837 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 140838 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 141842 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 141842 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 142846 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 142847 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 143851 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 143852 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 144855 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 144856 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 145859 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 145860 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 146863 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 146864 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 147867 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 147869 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 148871 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 148872 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 149875 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 149876 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 150880 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 150880 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 151884 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 151884 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 152889 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 152889 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 153893 T552 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 153893 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 153897 T552 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:144)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:204)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] 2> at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:131)
[junit4] 2> ... 10 more
[junit4] 2> Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:457)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:454)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:411)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
[junit4] 2> at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:136)
[junit4] 2> at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
[junit4] 2> ... 11 more
[junit4] 2>
[junit4] 2> 153899 T540 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 153900 T539 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 153903 T483 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34574 34574
[junit4] 2> 154022 T526 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:34574 34574
[junit4] 2> 154023 T483 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 154024 T549 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:34574_solr
[junit4] 2> 154025 T553 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=53BB0B78D4E89FC5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=ko_KR -Dtests.timezone=Indian/Antananarivo -Dtests.file.encoding=UTF-8
[junit4] ERROR 18.3s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
[junit4] > at __randomizedtesting.SeedInfo.seed([53BB0B78D4E89FC5:57B3848BC64D70E4]:0)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:144)
[junit4] > at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:155)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:660)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: org.apache.solr.common.SolrException: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] > at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:40)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:131)
[junit4] > ... 48 more
[junit4] > Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:457)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:454)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:411)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:398)
[junit4] > at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:136)
[junit4] > at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:34)
[junit4] > ... 49 more
[junit4] 2> 154115 T483 oas.SolrTestCaseJ4.setUp ###Starting testBadQueueItem
[junit4] 2> 154116 T483 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 154117 T554 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 154217 T483 oasc.ZkTestServer.run start zk server on port:17441
[junit4] 2> 154218 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 154221 T560 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d2c47e9 name:ZooKeeperConnection Watcher:127.0.0.1:17441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 154222 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 154224 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 154226 T562 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@427a26e2 name:ZooKeeperConnection Watcher:127.0.0.1:17441 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 154226 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 154226 T483 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 154230 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 154231 T564 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a6bdb60 name:ZooKeeperConnection Watcher:127.0.0.1:17441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 154232 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 154232 T483 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 154235 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 154236 T566 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64109ce7 name:ZooKeeperConnection Watcher:127.0.0.1:17441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 154236 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 154238 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 154240 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 154245 T483 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:17441_solr
[junit4] 2> 154245 T483 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 154247 T483 oasc.Overseer.start Overseer (id=92213279997493251-127.0.0.1:17441_solr-n_0000000000) starting
[junit4] 2> 154250 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 154253 T483 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 154256 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 154260 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 154263 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 154267 T483 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 154273 T568 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 154274 T483 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 154274 T567 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 154276 T483 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 154278 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 154281 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 154283 T570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d471f2d name:ZooKeeperConnection Watcher:127.0.0.1:17441/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 154283 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 154285 T483 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 154288 T483 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 154290 T571 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 154291 T572 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4] 2> 154293 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 154294 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 154295 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 154295 T567 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 154296 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 154299 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 154300 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 154300 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 154793 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 154803 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 154809 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 154811 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 154811 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 154815 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 154816 T567 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 154816 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 154818 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 154921 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 154921 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 155311 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 155319 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 155324 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155324 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155324 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155326 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155326 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 155330 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 155331 T567 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 155331 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 155333 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155436 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 155436 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 155826 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 155834 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 155840 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155840 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155840 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155841 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 155845 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155947 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 155947 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 155995 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155995 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155995 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155996 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 155997 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 155997 T567 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":"",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 156001 T567 oasc.Overseer$ClusterStateUpdater.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
[junit4] 2> "operation":"deletecore",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core0",
[junit4] 2> "core_node_name":"node0",
[junit4] 2> "collection":""}
[junit4] 2> 156004 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156005 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 156005 T567 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection2 with shards [shard1, shard2, shard3]
[junit4] 2> 156005 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 156008 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156110 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 156110 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 156499 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard3/election
[junit4] 2> 156508 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard3
[junit4] 2> 156514 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156514 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156514 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156516 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156516 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 156521 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 156521 T567 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 156521 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 156524 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 156626 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 156626 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 157016 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard2/election
[junit4] 2> 157024 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard2
[junit4] 2> 157029 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157029 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157029 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157031 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157031 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 157035 T567 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection2",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 157035 T567 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 157035 T567 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 157038 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157141 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 157141 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 157531 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard1/election
[junit4] 2> 157539 T483 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard1
[junit4] 2> 157544 T574 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157544 T575 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157544 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157545 T567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 157548 T573 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 157651 T571 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 157651 T572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 157699 T571 oascc.ZkStateReader$3.process Updating live nodes... (0)
[junit4] 2> 157701 T483 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:17441 17441
[junit4] 2> 157875 T554 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:17441 17441
[junit4] 2> 157876 T483 oas.SolrTestCaseJ4.tearDown ###Ending testBadQueueItem
[junit4] 2> 157876 T567 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:17441_solr
[junit4] 2> 157877 T576 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 157884 T483 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 157885 T483 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 157886 T577 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 157986 T483 oasc.ZkTestServer.run start zk server on port:25338
[junit4] 2> 157987 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 157990 T583 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64cbff21 name:ZooKeeperConnection Watcher:127.0.0.1:25338 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 157991 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 157993 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 157994 T585 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e353af0 name:ZooKeeperConnection Watcher:127.0.0.1:25338 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 157995 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 157995 T483 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 157999 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 158000 T587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3cfd7642 name:ZooKeeperConnection Watcher:127.0.0.1:25338/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 158000 T483 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 158001 T483 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 158003 T483 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 158004 T589 oascc.ConnectionManager.pro
[...truncated too long message...]
ndex [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest2_shard1_replica1/data/index;done=false>>]
[junit4] 2> 3780374 T10355 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest2_shard1_replica1/data/index
[junit4] 2> 3780374 T10355 oasc.SolrCore.close [ocptest_shardsplit_shard4_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@853f143
[junit4] 2> 3780383 T10355 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 3780383 T10355 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 3780384 T10355 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 3780384 T10355 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 3780385 T10355 oasc.SolrCore.closeSearcher [ocptest_shardsplit_shard4_replica1] Closing main searcher on request.
[junit4] 2> 3780385 T10355 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 3780385 T10355 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data/index [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data/index;done=false>>]
[junit4] 2> 3780386 T10355 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data/index
[junit4] 2> 3780386 T10355 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data;done=false>>]
[junit4] 2> 3780386 T10355 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001/tempDir-006/ocptest_shardsplit_shard4_replica1/data
[junit4] 2> 3780387 T11942 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:37404_nzjt%2Fpm
[junit4] 2> 3780388 T11944 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 3781888 T10469 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 3781890 T10355 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/nzjt/pm,null}
[junit4] 2> 3784332 T10355 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 3784334 T10355 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21266 21266
[junit4] 2> 3784477 T10356 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:21266 21266
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=53BB0B78D4E89FC5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=ko -Dtests.timezone=Antarctica/Casey -Dtests.file.encoding=UTF-8
[junit4] FAILURE 58.7s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([53BB0B78D4E89FC5:D25D8560A3B7FFF9]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3784487 T10355 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-53BB0B78D4E89FC5-001
[junit4] 2> 58682 T10354 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene49: {}, docValues:{}, sim=DefaultSimilarity, locale=ko, timezone=Antarctica/Casey
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=225480784,total=368574464
[junit4] 2> NOTE: All tests run in this JVM: [TestSurroundQueryParser, TestRandomMergePolicy, TestStressReorder, TestPartialUpdateDeduplication, PathHierarchyTokenizerFactoryTest, TestManagedSynonymFilterFactory, SuggesterWFSTTest, SignatureUpdateProcessorFactoryTest, DistributedSpellCheckComponentTest, TestRangeQuery, JsonLoaderTest, TestSolrDeletionPolicy1, OverseerTest, RollingRestartTest, ExternalFileFieldSortTest, TestBinaryResponseWriter, TestRemoteStreaming, TriLevelCompositeIdRoutingTest, CurrencyFieldOpenExchangeTest, XmlUpdateRequestHandlerTest, TestHashPartitioner, SpellPossibilityIteratorTest, TestDynamicFieldResource, TestComponentsName, TestReRankQParserPlugin, DateFieldTest, ReplicationFactorTest, TestFieldResource, StressHdfsTest, TestSimpleQParserPlugin, TestManagedSchema, TestLuceneMatchVersion, TestConfig, TestFieldTypeCollectionResource, CoreAdminHandlerTest, DocExpirationUpdateProcessorFactoryTest, CursorPagingTest, SyncSliceTest, TestCollapseQParserPlugin, TestRequestStatusCollectionAPI, BasicFunctionalityTest, TestComplexPhraseQParserPlugin, ParsingFieldUpdateProcessorsTest, TestCloudManagedSchemaConcurrent, ResourceLoaderTest, PrimUtilsTest, DocValuesMultiTest, TestFaceting, ZkControllerTest, MinimalSchemaTest, SimpleFacetsTest, UpdateParamsTest, TestStressUserVersions, BadCopyFieldTest, TestIntervalFaceting, SolrTestCaseJ4Test, MergeStrategyTest, CollectionsAPIAsyncDistributedZkTest, RegexBoostProcessorTest, StandardRequestHandlerTest, UnloadDistributedZkTest, TestStressVersions, TestDFRSimilarityFactory, OverseerCollectionProcessorTest, TestCollationKeyRangeQueries, TestLMJelinekMercerSimilarityFactory, TestFastOutputStream, TestNRTOpen, TestFieldTypeResource, PingRequestHandlerTest, TestSchemaResource, PluginInfoTest, IndexBasedSpellCheckerTest, TestManagedResource, SpellCheckCollatorTest, HdfsChaosMonkeySafeLeaderTest, BadIndexSchemaTest, TestSort, SpellCheckComponentTest, ZkCLITest, QueryEqualityTest, MultiThreadedOCPTest]
[junit4] Completed on J0 in 59.22s, 1 test, 1 failure <<< FAILURES!
[...truncated 695 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 411 suites, 1707 tests, 1 error, 1 failure, 50 ignored (8 assumptions)
Total time: 237 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 460095004 bytes
Compression is 0.0%
Took 1 min 18 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-NightlyTests-4.x - Build # 589 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-4.x/589/
1 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.testDistribSearch
Error Message:
We have a failed SPLITSHARD task
Stack Trace:
java.lang.AssertionError: We have a failed SPLITSHARD task
at __randomizedtesting.SeedInfo.seed([1498AB7BD9E81E86:957E2563AEB77EBA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12031 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/init-core-data-001
[junit4] 2> 2598738 T3771 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 2598739 T3771 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 2598742 T3771 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 2598743 T3771 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2598744 T3772 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2598844 T3771 oasc.ZkTestServer.run start zk server on port:20967
[junit4] 2> 2598845 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2598849 T3778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61d702b4 name:ZooKeeperConnection Watcher:127.0.0.1:20967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2598849 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2598850 T3771 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2598853 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2598854 T3780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46724c7a name:ZooKeeperConnection Watcher:127.0.0.1:20967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2598854 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2598854 T3771 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2598856 T3771 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2598858 T3771 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2598859 T3771 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2598861 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2598862 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2598865 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2598865 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2598868 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2598868 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2598870 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2598871 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2598873 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2598873 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2598875 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2598876 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2598878 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2598878 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2598880 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2598881 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2598883 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2598884 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2598885 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2598886 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2598888 T3771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2598889 T3771 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2600098 T3771 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2600101 T3771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:29655
[junit4] 2> 2600102 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2600102 T3771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2600102 T3771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002
[junit4] 2> 2600103 T3771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/'
[junit4] 2> 2600133 T3771 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/solr.xml
[junit4] 2> 2600185 T3771 oasc.CoreContainer.<init> New CoreContainer 1845206877
[junit4] 2> 2600186 T3771 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/]
[junit4] 2> 2600187 T3771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2600187 T3771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2600187 T3771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2600187 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2600188 T3771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2600188 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2600188 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2600188 T3771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2600188 T3771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2600189 T3771 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2600189 T3771 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2600189 T3771 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2600190 T3771 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2600190 T3771 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:20967/solr
[junit4] 2> 2600190 T3771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2600191 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2600192 T3791 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@229bcb14 name:ZooKeeperConnection Watcher:127.0.0.1:20967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2600193 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2600194 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2600195 T3793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c8d3f38 name:ZooKeeperConnection Watcher:127.0.0.1:20967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2600195 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2600196 T3771 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2600197 T3771 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2600199 T3771 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 2600200 T3771 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 2600201 T3771 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 2600203 T3771 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2600204 T3771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:29655_
[junit4] 2> 2600204 T3771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:29655_
[junit4] 2> 2600205 T3771 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2600206 T3771 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2600208 T3771 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:29655_
[junit4] 2> 2600208 T3771 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2600209 T3771 oasc.Overseer.start Overseer (id=92207517014032387-127.0.0.1:29655_-n_0000000000) starting
[junit4] 2> 2600211 T3771 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2600215 T3795 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2600215 T3771 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2600216 T3771 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2600217 T3771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2600218 T3794 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2600222 T3796 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 2600222 T3796 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2600222 T3796 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2600223 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2600223 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2600224 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:29655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:29655_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"1",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2600224 T3794 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 2600224 T3794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2600225 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2600226 T3797 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2601223 T3796 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2601224 T3796 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 2601225 T3796 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2601225 T3796 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2601226 T3796 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2601226 T3796 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/collection1/'
[junit4] 2> 2601227 T3796 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 2601228 T3796 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 2601228 T3796 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 2601291 T3796 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 2601335 T3796 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2601336 T3796 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2601346 T3796 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2601745 T3796 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2601746 T3796 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2601746 T3796 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2601759 T3796 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2601762 T3796 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2601774 T3796 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2601778 T3796 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2601788 T3796 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2601789 T3796 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2601789 T3796 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2601789 T3796 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2601790 T3796 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2601790 T3796 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2601790 T3796 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2601790 T3796 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2601791 T3796 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-002/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-001/control/data/
[junit4] 2> 2601791 T3796 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34ae40d3
[junit4] 2> 2601792 T3796 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-001/control/data
[junit4] 2> 2601792 T3796 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-001/control/data/index/
[junit4] 2> 2601792 T3796 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 2601793 T3796 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-001/control/data/index
[junit4] 2> 2601793 T3796 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=13.591796875, floorSegmentMB=2.1748046875, forceMergeDeletesPctAllowed=21.363720158407794, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2185607102173476
[junit4] 2> 2601794 T3796 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@53b85955 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4e4e3aaa)),segFN=segments_1,generation=1}
[junit4] 2> 2601794 T3796 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2601798 T3796 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2601798 T3796 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2601798 T3796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2601799 T3796 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2601799 T3796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2601799 T3796 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2601800 T3796 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2601800 T3796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2601800 T3796 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2601801 T3796 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2601801 T3796 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2601801 T3796 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2601802 T3796 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2601802 T3796 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2601802 T3796 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2601803 T3796 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2601821 T3796 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2601827 T3796 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2601827 T3796 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2601828 T3796 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=35.126953125, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=4.5515110665894065, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26227905101379545
[junit4] 2> 2601828 T3796 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@53b85955 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4e4e3aaa)),segFN=segments_1,generation=1}
[junit4] 2> 2601828 T3796 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2601829 T3796 oass.SolrIndexSearcher.<init> Opening Searcher@5009611b[collection1] main
[junit4] 2> 2601829 T3796 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2601830 T3796 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2601830 T3796 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2601830 T3796 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2601831 T3796 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2601831 T3796 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2601831 T3796 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 2601832 T3796 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2601832 T3796 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 2601834 T3796 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 2601834 T3796 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2601835 T3796 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2601837 T3798 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5009611b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2601839 T3796 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2601840 T3801 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:29655 collection:control_collection shard:shard1
[junit4] 2> 2601840 T3771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 2601841 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2601841 T3801 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 2601842 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2601843 T3803 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e8616d8 name:ZooKeeperConnection Watcher:127.0.0.1:20967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2601844 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2601844 T3771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2601846 T3801 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2601846 T3771 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2601847 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2601847 T3801 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2601847 T3801 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1250 name=collection1 org.apache.solr.core.SolrCore@6b9b70d0 url=http://127.0.0.1:29655/collection1 node=127.0.0.1:29655_ C1250_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:29655, core=collection1, node_name=127.0.0.1:29655_}
[junit4] 2> 2601848 T3801 C1250 P29655 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:29655/collection1/
[junit4] 2> 2601848 T3801 C1250 P29655 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2601848 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2601848 T3801 C1250 P29655 oasc.SyncStrategy.syncToMe http://127.0.0.1:29655/collection1/ has no replicas
[junit4] 2> 2601849 T3801 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:29655/collection1/ shard1
[junit4] 2> 2601849 T3801 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 2601850 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2601851 T3797 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2601851 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2601854 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2601855 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2601957 T3797 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2601957 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2602006 T3801 oasc.ZkController.register We are http://127.0.0.1:29655/collection1/ and leader is http://127.0.0.1:29655/collection1/
[junit4] 2> 2602007 T3801 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:29655
[junit4] 2> 2602007 T3801 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2602007 T3801 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 2602007 T3801 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2602008 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2602008 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2602009 T3801 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2602009 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2602010 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2602010 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:29655",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:29655_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2602012 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2602113 T3806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2602113 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2602950 T3771 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 2602951 T3771 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2602954 T3771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:47424
[junit4] 2> 2602954 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2602954 T3771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2602954 T3771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003
[junit4] 2> 2602955 T3771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/'
[junit4] 2> 2602984 T3771 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/solr.xml
[junit4] 2> 2603035 T3771 oasc.CoreContainer.<init> New CoreContainer 1908158125
[junit4] 2> 2603035 T3771 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/]
[junit4] 2> 2603036 T3771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2603036 T3771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2603036 T3771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2603037 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2603037 T3771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2603037 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2603037 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2603037 T3771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2603038 T3771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2603038 T3771 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2603038 T3771 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2603039 T3771 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2603039 T3771 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2603039 T3771 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:20967/solr
[junit4] 2> 2603039 T3771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2603040 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2603041 T3817 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@799c9a30 name:ZooKeeperConnection Watcher:127.0.0.1:20967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2603042 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2603044 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2603044 T3819 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62776e5c name:ZooKeeperConnection Watcher:127.0.0.1:20967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2603045 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2603049 T3771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2604051 T3771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47424_
[junit4] 2> 2604052 T3771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47424_
[junit4] 2> 2604054 T3806 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2604055 T3820 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2604054 T3804 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4] 2> 2604061 T3821 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2604062 T3821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2604062 T3821 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2604062 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2604062 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2604062 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2604064 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2604064 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:47424",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47424_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2604065 T3794 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 2604065 T3794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 2604067 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2604067 T3806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2604067 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2604067 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2605063 T3821 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2605064 T3821 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2605065 T3821 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2605065 T3821 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2605066 T3821 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2605066 T3821 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/'
[junit4] 2> 2605085 T3821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 2605092 T3821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 2605099 T3821 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 2605406 T3821 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 2605516 T3821 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2605518 T3821 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2605538 T3821 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2606493 T3821 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2606494 T3821 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2606495 T3821 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2606530 T3821 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2606537 T3821 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2606561 T3821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2606570 T3821 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2606579 T3821 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2606581 T3821 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2606581 T3821 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2606582 T3821 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2606583 T3821 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2606583 T3821 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2606584 T3821 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2606584 T3821 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2606584 T3821 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/, dataDir=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/data/
[junit4] 2> 2606585 T3821 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34ae40d3
[junit4] 2> 2606586 T3821 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/data
[junit4] 2> 2606587 T3821 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/data/index/
[junit4] 2> 2606587 T3821 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 2606587 T3821 oasc.CachingDirectoryFactory.get return new directory for ../../../../../../../../../../../usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-003/collection1/data/index
[junit4] 2> 2606588 T3821 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=13.591796875, floorSegmentMB=2.1748046875, forceMergeDeletesPctAllowed=21.363720158407794, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2185607102173476
[junit4] 2> 2606589 T3821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@364c8ab9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@275c359d)),segFN=segments_1,generation=1}
[junit4] 2> 2606590 T3821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2606595 T3821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2606596 T3821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2606596 T3821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2606596 T3821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2606597 T3821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2606597 T3821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2606598 T3821 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2606599 T3821 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2606599 T3821 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2606600 T3821 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2606601 T3821 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2606601 T3821 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2606602 T3821 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2606602 T3821 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2606603 T3821 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2606604 T3821 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2606657 T3821 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2606673 T3821 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2606673 T3821 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2606674 T3821 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=35.126953125, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=4.5515110665894065, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26227905101379545
[junit4] 2> 2606675 T3821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@364c8ab9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@275c359d)),segFN=segments_1,generation=1}
[junit4] 2> 2606675 T3821 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2606675 T3821 oass.SolrIndexSearcher.<init> Opening Searcher@4d95fdf8[collection1] main
[junit4] 2> 2606676 T3821 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2606677 T3821 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2606677 T3821 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2606678 T3821 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2606679 T3821 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2606679 T3821 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2606680 T3821 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 2606681 T3821 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2606681 T3821 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 2606681 T3821 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2606687 T3822 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d95fdf8[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2606689 T3821 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2606690 T3825 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47424 collection:collection1 shard:shard2
[junit4] 2> 2606691 T3771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 2606691 T3825 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 2606691 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2606706 T3825 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 2606711 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606712 T3825 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2606713 T3825 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1251 name=collection1 org.apache.solr.core.SolrCore@7f132c21 url=http://127.0.0.1:47424/collection1 node=127.0.0.1:47424_ C1251_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:47424, core=collection1, node_name=127.0.0.1:47424_}
[junit4] 2> 2606713 T3825 C1251 P47424 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:47424/collection1/
[junit4] 2> 2606713 T3825 C1251 P47424 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2606714 T3825 C1251 P47424 oasc.SyncStrategy.syncToMe http://127.0.0.1:47424/collection1/ has no replicas
[junit4] 2> 2606714 T3825 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:47424/collection1/ shard2
[junit4] 2> 2606714 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2606715 T3825 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 2606724 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606726 T3806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606727 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606727 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606733 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2606737 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606842 T3806 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606842 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606843 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606884 T3825 oasc.ZkController.register We are http://127.0.0.1:47424/collection1/ and leader is http://127.0.0.1:47424/collection1/
[junit4] 2> 2606885 T3825 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47424
[junit4] 2> 2606885 T3825 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2606885 T3825 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 2606886 T3825 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2606888 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606888 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606888 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606889 T3825 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2606890 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2606891 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:47424",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:47424_",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node1"}
[junit4] 2> 2606894 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2606996 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606996 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2606996 T3805 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2608243 T3771 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 2608244 T3771 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2608247 T3771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16313
[junit4] 2> 2608247 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2608248 T3771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2608248 T3771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004
[junit4] 2> 2608248 T3771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/'
[junit4] 2> 2608283 T3771 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/solr.xml
[junit4] 2> 2608335 T3771 oasc.CoreContainer.<init> New CoreContainer 549509802
[junit4] 2> 2608335 T3771 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/]
[junit4] 2> 2608336 T3771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2608336 T3771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2608337 T3771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2608337 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2608337 T3771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2608337 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2608338 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2608338 T3771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2608338 T3771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2608338 T3771 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2608339 T3771 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2608339 T3771 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2608339 T3771 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2608339 T3771 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:20967/solr
[junit4] 2> 2608340 T3771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2608340 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2608342 T3836 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5553dfb2 name:ZooKeeperConnection Watcher:127.0.0.1:20967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2608343 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2608344 T3771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2608345 T3838 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16f1b09e name:ZooKeeperConnection Watcher:127.0.0.1:20967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2608345 T3771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2608350 T3771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2609353 T3771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16313_
[junit4] 2> 2609354 T3771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16313_
[junit4] 2> 2609357 T3804 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2609357 T3805 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2609357 T3820 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2609357 T3839 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4] 2> 2609364 T3840 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2609364 T3840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2609365 T3840 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2609365 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2609365 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2609365 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2609366 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2609367 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:16313",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:16313_",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":null}
[junit4] 2> 2609367 T3794 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2609367 T3794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2609369 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2609370 T3805 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2609370 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2609370 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2609370 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2610365 T3840 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2610366 T3840 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2610366 T3840 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2610366 T3840 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2610367 T3840 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2610367 T3840 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/'
[junit4] 2> 2610369 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 2610369 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 2610370 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 2610429 T3840 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
[junit4] 2> 2610485 T3840 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2610486 T3840 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2610496 T3840 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2610910 T3840 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2610911 T3840 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4] 2> 2610912 T3840 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2610926 T3840 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2610929 T3840 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2610941 T3840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2610945 T3840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2610949 T3840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2610950 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2610950 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2610950 T3840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2610951 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2610951 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2610951 T3840 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2610952 T3840 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2610952 T3840 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/, dataDir=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/data/
[junit4] 2> 2610952 T3840 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34ae40d3
[junit4] 2> 2610953 T3840 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/data
[junit4] 2> 2610953 T3840 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/data/index/
[junit4] 2> 2610953 T3840 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/data/index' doesn't exist. Creating new index...
[junit4] 2> 2610954 T3840 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-004/collection1/data/index
[junit4] 2> 2610954 T3840 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=13.591796875, floorSegmentMB=2.1748046875, forceMergeDeletesPctAllowed=21.363720158407794, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2185607102173476
[junit4] 2> 2610955 T3840 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@5b588420 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1eb2f61f)),segFN=segments_1,generation=1}
[junit4] 2> 2610955 T3840 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2610960 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2610960 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2610961 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2610961 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2610961 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2610961 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2610962 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2610962 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2610963 T3840 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2610963 T3840 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2610964 T3840 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2610964 T3840 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2610964 T3840 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2610965 T3840 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2610965 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2610966 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2610987 T3840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2610992 T3840 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2610992 T3840 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2610993 T3840 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=35.126953125, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=4.5515110665894065, segmentsPerTier=32.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.26227905101379545
[junit4] 2> 2610994 T3840 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RateLimitedDirectoryWrapper(RAMDirectory@5b588420 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1eb2f61f)),segFN=segments_1,generation=1}
[junit4] 2> 2610994 T3840 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2610994 T3840 oass.SolrIndexSearcher.<init> Opening Searcher@4120b6fe[collection1] main
[junit4] 2> 2610995 T3840 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2610995 T3840 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2610996 T3840 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2610996 T3840 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2610996 T3840 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2610997 T3840 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2610997 T3840 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 2610998 T3840 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2610998 T3840 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 2610998 T3840 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2611002 T3841 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4120b6fe[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4] 2> 2611004 T3840 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2611005 T3844 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16313 collection:collection1 shard:shard1
[junit4] 2> 2611005 T3771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0
[junit4] 2> 2611006 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2611006 T3844 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2611010 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2611011 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611011 T3844 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2611012 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C1252 name=collection1 org.apache.solr.core.SolrCore@576dbe87 url=http://127.0.0.1:16313/collection1 node=127.0.0.1:16313_ C1252_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:16313, core=collection1, node_name=127.0.0.1:16313_}
[junit4] 2> 2611012 T3844 C1252 P16313 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16313/collection1/
[junit4] 2> 2611012 T3844 C1252 P16313 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2611012 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2611012 T3844 C1252 P16313 oasc.SyncStrategy.syncToMe http://127.0.0.1:16313/collection1/ has no replicas
[junit4] 2> 2611013 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16313/collection1/ shard1
[junit4] 2> 2611013 T3844 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2611014 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611015 T3805 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611015 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611015 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611015 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611018 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2611020 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611121 T3805 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611122 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611121 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611121 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611169 T3844 oasc.ZkController.register We are http://127.0.0.1:16313/collection1/ and leader is http://127.0.0.1:16313/collection1/
[junit4] 2> 2611170 T3844 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16313
[junit4] 2> 2611170 T3844 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2611170 T3844 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 2611170 T3844 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2611171 T3805 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611172 T3844 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2611172 T3806 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611171 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611173 T3794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4] 2> 2611174 T3794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "base_url":"http://127.0.0.1:16313",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "node_name":"127.0.0.1:16313_",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "core_node_name":"core_node2"}
[junit4] 2> 2611175 T3797 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2611278 T3804 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611278 T3820 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611278 T3797 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2611278 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2612059 T3771 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 2612060 T3771 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2612062 T3771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16338
[junit4] 2> 2612062 T3771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2612063 T3771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2612063 T3771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-005
[junit4] 2> 2612063 T3771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-005/'
[junit4] 2> 2612092 T3771 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-005/solr.xml
[junit4] 2> 2612141 T3771 oasc.CoreContainer.<init> New CoreContainer 100020689
[junit4] 2> 2612142 T3771 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-005/]
[junit4] 2> 2612143 T3771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2612143 T3771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2612143 T3771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2612143 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2612144 T3771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2612144 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2612144 T3771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2612144 T3771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2612144 T3771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2612145 T3771 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTi
[...truncated too long message...]
[CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest2_shard3_replica1/data/index;done=false>>]
[junit4] 2> 2648984 T3771 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest2_shard3_replica1/data/index
[junit4] 2> 2648985 T3771 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest2_shard3_replica1/data [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest2_shard3_replica1/data;done=false>>]
[junit4] 2> 2648985 T3771 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest2_shard3_replica1/data
[junit4] 2> 2648985 T3771 oasc.SolrCore.close [ocptest3_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@7ac26668
[junit4] 2> 2648993 T3771 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 2648993 T3771 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 2648994 T3771 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 2648994 T3771 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 2648995 T3771 oasc.SolrCore.closeSearcher [ocptest3_shard1_replica1] Closing main searcher on request.
[junit4] 2> 2648995 T3771 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 2648995 T3771 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data;done=false>>]
[junit4] 2> 2648996 T3771 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data
[junit4] 2> 2648996 T3771 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data/index [CachedDir<<refCount=0;path=./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data/index;done=false>>]
[junit4] 2> 2648996 T3771 oasc.CachingDirectoryFactory.close Closing directory: ./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001/tempDir-006/ocptest3_shard1_replica1/data/index
[junit4] 2> 2648997 T5321 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:16342_
[junit4] 2> 2648999 T5316 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2648999 T5323 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:302)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:299)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
[junit4] 2>
[junit4] 2> 2649000 T3771 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 2653958 T3771 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 2653960 T3771 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:20967 20967
[junit4] 2> 2654153 T3772 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:20967 20967
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=testDistribSearch -Dtests.seed=1498AB7BD9E81E86 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/hudson/lucene-data/enwiki.random.lines.txt -Dtests.locale=es_AR -Dtests.timezone=Asia/Anadyr -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 55.4s J0 | MultiThreadedOCPTest.testDistribSearch <<<
[junit4] > Throwable #1: java.lang.AssertionError: We have a failed SPLITSHARD task
[junit4] > at __randomizedtesting.SeedInfo.seed([1498AB7BD9E81E86:957E2563AEB77EBA]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testTaskExclusivity(MultiThreadedOCPTest.java:125)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.doTest(MultiThreadedOCPTest.java:71)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2654176 T3771 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build/solr-core/test/J0/./solr.cloud.MultiThreadedOCPTest-1498AB7BD9E81E86-001
[junit4] 2> 55443 T3770 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene46, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_AR, timezone=Asia/Anadyr
[junit4] 2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=189123928,total=333971456
[junit4] 2> NOTE: All tests run in this JVM: [TestStressReorder, EchoParamsTest, TestSearcherReuse, TestDFRSimilarityFactory, PluginInfoTest, TestCodecSupport, TestCloudManagedSchema, TestCloudSchemaless, BasicDistributedZkTest, TestComponentsName, WordBreakSolrSpellCheckerTest, ZkCLITest, HdfsCollectionsAPIDistributedZkTest, AnalyticsQueryTest, TestFastOutputStream, TestRandomMergePolicy, SuggesterWFSTTest, TestSolrXmlPersistence, TestMiniSolrCloudCluster, RequiredFieldsTest, TestFaceting, DirectSolrSpellCheckerTest, PathHierarchyTokenizerFactoryTest, ReplicationFactorTest, SolrCmdDistributorTest, TestHashPartitioner, TestTrie, TestRangeQuery, MBeansHandlerTest, PingRequestHandlerTest, TestComplexPhraseQParserPlugin, TestSolrDeletionPolicy1, MinimalSchemaTest, TestStandardQParsers, XsltUpdateRequestHandlerTest, FieldMutatingUpdateProcessorTest, TestBM25SimilarityFactory, TestStressVersions, TestManagedResource, SliceStateTest, TestSimpleQParserPlugin, TestClassNameShortening, TestFuzzyAnalyzedSuggestions, TestIntervalFaceting, BadCopyFieldTest, SpellPossibilityIteratorTest, SolrTestCaseJ4Test, TestMaxScoreQueryParser, TestManagedSynonymFilterFactory, TestRestManager, SimpleFacetsTest, StandardRequestHandlerTest, MergeStrategyTest, CurrencyFieldOpenExchangeTest, QueryEqualityTest, TestUtils, BadIndexSchemaTest, TestFieldResource, SolrXmlInZkTest, DistributedDebugComponentTest, ExternalFileFieldSortTest, TestSurroundQueryParser, NoCacheHeaderTest, TestSchemaResource, CollectionsAPIAsyncDistributedZkTest, TestNumberUtils, TestStressRecovery, MultiThreadedOCPTest]
[junit4] Completed on J0 in 55.97s, 1 test, 1 failure <<< FAILURES!
[...truncated 763 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:474: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-NightlyTests-4.x/lucene/common-build.xml:923: There were test failures: 411 suites, 1705 tests, 1 failure, 50 ignored (8 assumptions)
Total time: 228 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-NightlyTests-4.x #586
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 675968823 bytes
Compression is 0.0%
Took 1 min 51 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure