You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/05/15 14:25:15 UTC

[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2307 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2307/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Error from server at http://127.0.0.1:62194//collection1: java.lang.NullPointerException  at org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:102)  at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:744)  at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:727)  at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2047)  at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:636)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:198)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:178)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)  at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  at org.eclipse.jetty.server.Server.handle(Server.java:497)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)  at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)  at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)  at java.lang.Thread.run(Thread.java:745) 

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62194//collection1: java.lang.NullPointerException
	at org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:102)
	at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:744)
	at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:727)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2047)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:636)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:198)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:178)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:497)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)

	at __randomizedtesting.SeedInfo.seed([47AE2BD9E5661547:CFFA14034B9A78BF]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
	at org.apache.solr.TestDistributedSearch.queryRandomUpServer(TestDistributedSearch.java:1133)
	at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1095)
	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:968)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:982)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10135 lines...]
   [junit4] Suite: org.apache.solr.TestDistributedSearch
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/init-core-data-001
   [junit4]   2> 1577936 T8131 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1577936 T8131 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1578358 T8131 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1
   [junit4]   2> 1578362 T8131 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1578367 T8131 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@361aca79{/,null,AVAILABLE}
   [junit4]   2> 1578389 T8131 oejs.AbstractConnector.doStart Started ServerConnector@19123d79{HTTP/1.1}{127.0.0.1:62187}
   [junit4]   2> 1578390 T8131 oejs.Server.doStart Started @1581198ms
   [junit4]   2> 1578390 T8131 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {hostContext=/, hostPort=62187, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores}
   [junit4]   2> 1578390 T8131 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@74a14482
   [junit4]   2> 1578391 T8131 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/'
   [junit4]   2> 1578433 T8131 oasc.SolrXmlConfig.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/solr.xml
   [junit4]   2> 1578448 T8131 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores
   [junit4]   2> 1578449 T8131 oasc.CoreContainer.<init> New CoreContainer 1455163053
   [junit4]   2> 1578449 T8131 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/]
   [junit4]   2> 1578449 T8131 oasc.CoreContainer.load loading shared library: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/lib
   [junit4]   2> 1578450 T8131 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/lib).
   [junit4]   2> 1578470 T8131 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1578487 T8131 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1578488 T8131 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1578488 T8131 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1578489 T8131 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1578493 T8131 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores
   [junit4]   2> 1578495 T8131 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1, absoluteInstDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/, configSet=collection1, dataDir=data/}
   [junit4]   2> 1578496 T8131 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/
   [junit4]   2> 1578496 T8131 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1578507 T8143 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1578508 T8143 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1578509 T8143 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1578555 T8143 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1578567 T8143 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1578594 T8143 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1578595 T8143 oass.IndexSchema.readSchema Reading Solr Schema from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1578601 T8143 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1578716 T8143 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1578726 T8143 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1578727 T8143 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1578732 T8143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1578734 T8143 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1578736 T8143 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1578738 T8143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1578738 T8143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1578738 T8143 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1578740 T8143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1578740 T8143 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1578741 T8143 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from configset /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1578741 T8143 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1578741 T8143 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 1578741 T8143 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d2290f0
   [junit4]   2> 1578742 T8143 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/data
   [junit4]   2> 1578742 T8143 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/data/index/
   [junit4]   2> 1578743 T8143 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1578743 T8143 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/control/cores/collection1/data/index
   [junit4]   2> 1578743 T8143 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=83.546875, floorSegmentMB=2.1962890625, forceMergeDeletesPctAllowed=2.6965847398841056, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1578744 T8143 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_1,generation=1}
   [junit4]   2> 1578745 T8143 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1578752 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1578753 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1578753 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1578753 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1578753 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1578754 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1578754 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1578754 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1578754 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1578755 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1578755 T8143 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1578755 T8143 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1578765 T8143 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1578768 T8143 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1578771 T8143 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1578773 T8143 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1578780 T8143 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1578783 T8143 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 1578784 T8143 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 1578785 T8143 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1578787 T8143 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1578787 T8143 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 1578789 T8143 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1578789 T8143 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1578790 T8143 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=54.3671875, floorSegmentMB=1.1552734375, forceMergeDeletesPctAllowed=7.974779601161448, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4756950038835368
   [junit4]   2> 1578790 T8143 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_1,generation=1}
   [junit4]   2> 1578790 T8143 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1578791 T8143 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@4d77b6f5[collection1] main
   [junit4]   2> 1578791 T8143 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 1578792 T8143 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1578792 T8143 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1578793 T8143 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1578806 T8143 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1578857 T8143 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1578858 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 1578859 T8143 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1578859 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
   [junit4]   2> 1578860 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1578860 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1578860 T8131 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1
   [junit4]   2> 1578861 T8131 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1578862 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1578864 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
   [junit4]   2> 1578870 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
   [junit4]   2> 1578871 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
   [junit4]   2> 1578872 T8144 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
   [junit4]   2> 1578874 T8144 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d77b6f5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1579344 T8131 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 1579347 T8131 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 1579351 T8131 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@542267e6{/,null,AVAILABLE}
   [junit4]   2> 1579352 T8131 oejs.AbstractConnector.doStart Started ServerConnector@3ace96b{HTTP/1.1}{127.0.0.1:62188}
   [junit4]   2> 1579352 T8131 oejs.Server.doStart Started @1582161ms
   [junit4]   2> 1579353 T8131 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {hostContext=/, hostPort=62188, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores}
   [junit4]   2> 1579353 T8131 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@74a14482
   [junit4]   2> 1579353 T8131 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/'
   [junit4]   2> 1579404 T8131 oasc.SolrXmlConfig.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 1579420 T8131 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores
   [junit4]   2> 1579420 T8131 oasc.CoreContainer.<init> New CoreContainer 1134068460
   [junit4]   2> 1579421 T8131 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/]
   [junit4]   2> 1579421 T8131 oasc.CoreContainer.load loading shared library: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/lib
   [junit4]   2> 1579421 T8131 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/lib).
   [junit4]   2> 1579433 T8131 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 1579438 T8131 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1579439 T8131 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1579439 T8131 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1579439 T8131 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 1579441 T8131 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores
   [junit4]   2> 1579442 T8131 oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1, absoluteInstDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/, configSet=collection1, dataDir=data/}
   [junit4]   2> 1579442 T8131 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/
   [junit4]   2> 1579443 T8131 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1579444 T8157 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 1579445 T8157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 1579445 T8157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 1579475 T8157 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 1579483 T8157 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 1579501 T8157 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1579502 T8157 oass.IndexSchema.readSchema Reading Solr Schema from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml
   [junit4]   2> 1579507 T8157 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1579619 T8157 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1579637 T8157 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1579639 T8157 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1579646 T8157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1579649 T8157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1579651 T8157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1579653 T8157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1579653 T8157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1579653 T8157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1579653 T8157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1579653 T8157 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1579654 T8157 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from configset /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1579654 T8157 x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1579654 T8157 x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/], dataDir=[null]
   [junit4]   2> 1579654 T8157 x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d2290f0
   [junit4]   2> 1579655 T8157 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/data
   [junit4]   2> 1579655 T8157 x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/data/index/
   [junit4]   2> 1579655 T8157 x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1579655 T8157 x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001/tempDir-001/shard0/cores/collection1/data/index
   [junit4]   2> 1579655 T8157 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=83.546875, floorSegmentMB=2.1962890625, forceMergeDeletesPctAllowed=2.6965847398841056, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1579656 T8157 x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_1,generation=1}
   [junit4]   2> 1579656 T8157 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1579661 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1579662 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1579662 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 1579663 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1579664 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1579664 T8157 x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1579664 T8157 x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1579670 T8157 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1579671 T8157 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1579673 T8157 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1579674 T8157 x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1579681 T8157 x:collection1 oasc.PluginBag.createPlugin requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 1579685 T8157 x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 1579687 T8157 x:collection1 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 1579688 T8157 x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1579690 T8157 x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1579691 T8157 x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 1579692 T8157 x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1579693 T8157 x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1579694 T8157 x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=54.3671875, floorSegmentMB=1.1552734375, forceMergeDeletesPctAllowed=7.974779601161448, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4756950038835368
   [junit4]   2> 1579694 T8157 x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_1,generation=1}
   [junit4]   2> 1579695 T8157 x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1579695 T8157 x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@5604aec9[collection1] main
   [junit4]   2> 1579696 T8157 x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 1579696 T8157 x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 1579697 T8157 x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1579697 T8157 x:collection1 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 1579705 T8157 x:collection1 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1579747 T8157 x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 1579747 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: default
   [junit4]   2> 1579748 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: direct
   [junit4]   2> 1579748 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1579748 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1579748 T8157 x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1579749 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1579749 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: external
   [junit4]   2> 1579750 T8131 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1
   [junit4]   2> 1579750 T8131 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1579750 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: freq
   [junit4]   2> 1579752 T8131 oas.SolrTestCaseJ4.setUp ###Starting test
   [junit4]   2> 1579752 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: fqcn
   [junit4]   2> 1579753 T8158 x:collection1 oashc.SpellCheckComponent$SpellCheckerListener.newSearcher Loading spell index for spellchecker: perDict
   [junit4]   2> 1579756 T8158 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5604aec9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> ASYNC  NEW_CORE C1179 name=collection1 org.apache.solr.core.SolrCore@5ff06513
   [junit4]   2> 1579773 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1501237434873020416)} 0 3
   [junit4]   2> ASYNC  NEW_CORE C1180 name=collection1 org.apache.solr.core.SolrCore@3803d3e0
   [junit4]   2> 1579782 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1501237434881409024)} 0 4
   [junit4]   2> 1579789 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1501237434889797632)]} 0 3
   [junit4]   2> 1579796 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1501237434898186240)]} 0 3
   [junit4]   2> 1579801 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1501237434904477696)]} 0 2
   [junit4]   2> 1579805 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1501237434908672000)]} 0 1
   [junit4]   2> 1579810 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1501237434913914880)]} 0 2
   [junit4]   2> 1579818 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1501237434922303488)]} 0 2
   [junit4]   2> 1579832 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1501237434927546368)]} 0 10
   [junit4]   2> 1579838 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1501237434942226432)]} 0 3
   [junit4]   2> 1579844 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1501237434949566464)]} 0 1
   [junit4]   2> 1579849 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1501237434954809344)]} 0 2
   [junit4]   2> 1579859 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1501237434963197952)]} 0 4
   [junit4]   2> 1579864 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1501237434969489408)]} 0 2
   [junit4]   2> 1579895 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1501237435003043840)]} 0 1
   [junit4]   2> 1579901 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1501237435009335296)]} 0 1
   [junit4]   2> 1579909 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1501237435017723904)]} 0 2
   [junit4]   2> 1579923 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1501237435032403968)]} 0 2
   [junit4]   2> 1579932 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1501237435041841152)]} 0 1
   [junit4]   2> 1579939 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1501237435047084032)]} 0 3
   [junit4]   2> 1579943 T8139 x:collection1 C1179 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1579958 T8139 x:collection1 C1179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_2,generation=2}
   [junit4]   2> 1579960 T8139 x:collection1 C1179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1579962 T8139 x:collection1 C1179 oass.SolrIndexSearcher.<init> Opening Searcher@582e64b5[collection1] main
   [junit4]   2> 1579962 T8139 x:collection1 C1179 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1579963 T8144 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@582e64b5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c9)))}
   [junit4]   2> 1579963 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 20
   [junit4]   2> 1579975 T8153 x:collection1 C1180 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1579980 T8153 x:collection1 C1180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_2,generation=2}
   [junit4]   2> 1579981 T8153 x:collection1 C1180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 1579982 T8153 x:collection1 C1180 oass.SolrIndexSearcher.<init> Opening Searcher@5dcb1dc[collection1] main
   [junit4]   2> 1579982 T8153 x:collection1 C1180 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1579983 T8158 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5dcb1dc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c9)))}
   [junit4]   2> 1579984 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 8
   [junit4]   2> 1579990 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1501237435100561408)]} 0 4
   [junit4]   2> 1579997 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1501237435106852864)]} 0 4
   [junit4]   2> 1580001 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1501237435114192896)]} 0 1
   [junit4]   2> 1580006 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1501237435119435776)]} 0 2
   [junit4]   2> 1580010 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1501237435122581504)]} 0 2
   [junit4]   2> 1580013 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1501237435126775808)]} 0 1
   [junit4]   2> 1580017 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1501237435130970112)]} 0 2
   [junit4]   2> 1580021 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1501237435135164416)]} 0 1
   [junit4]   2> 1580025 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1001 (1501237435139358720)]} 0 1
   [junit4]   2> 1580029 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1001 (1501237435143553024)]} 0 1
   [junit4]   2> 1580032 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1501237435146698752)]} 0 1
   [junit4]   2> 1580036 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1501237435150893056)]} 0 1
   [junit4]   2> 1580041 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1501237435156135936)]} 0 2
   [junit4]   2> 1580046 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1501237435161378816)]} 0 1
   [junit4]   2> 1580050 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1501237435165573120)]} 0 0
   [junit4]   2> 1580053 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1501237435168718848)]} 0 1
   [junit4]   2> 1580057 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1501237435172913152)]} 0 2
   [junit4]   2> 1580061 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1501237435178156032)]} 0 1
   [junit4]   2> 1580069 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[100 (1501237435182350336)]} 0 4
   [junit4]   2> 1580073 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[100 (1501237435189690368)]} 0 1
   [junit4]   2> 1580077 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[101 (1501237435193884672)]} 0 1
   [junit4]   2> 1580081 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[101 (1501237435198078976)]} 0 1
   [junit4]   2> 1580084 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[102 (1501237435202273280)]} 0 1
   [junit4]   2> 1580088 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[102 (1501237435205419008)]} 0 1
   [junit4]   2> 1580092 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[103 (1501237435208564736)]} 0 2
   [junit4]   2> 1580095 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[103 (1501237435212759040)]} 0 1
   [junit4]   2> 1580098 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[104 (1501237435215904768)]} 0 1
   [junit4]   2> 1580101 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[104 (1501237435219050496)]} 0 1
   [junit4]   2> 1580105 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[105 (1501237435223244800)]} 0 1
   [junit4]   2> 1580108 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[105 (1501237435226390528)]} 0 1
   [junit4]   2> 1580112 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[106 (1501237435230584832)]} 0 1
   [junit4]   2> 1580116 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[106 (1501237435234779136)]} 0 1
   [junit4]   2> 1580120 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[107 (1501237435238973440)]} 0 1
   [junit4]   2> 1580123 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[107 (1501237435242119168)]} 0 1
   [junit4]   2> 1580129 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[108 (1501237435248410624)]} 0 1
   [junit4]   2> 1580132 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[108 (1501237435252604928)]} 0 0
   [junit4]   2> 1580136 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[109 (1501237435255750656)]} 0 1
   [junit4]   2> 1580140 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[109 (1501237435259944960)]} 0 1
   [junit4]   2> 1580143 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[110 (1501237435263090688)]} 0 1
   [junit4]   2> 1580146 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[110 (1501237435267284992)]} 0 1
   [junit4]   2> 1580150 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[111 (1501237435271479296)]} 0 1
   [junit4]   2> 1580154 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[111 (1501237435274625024)]} 0 1
   [junit4]   2> 1580158 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[112 (1501237435278819328)]} 0 1
   [junit4]   2> 1580165 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[112 (1501237435284062208)]} 0 4
   [junit4]   2> 1580169 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[113 (1501237435290353664)]} 0 1
   [junit4]   2> 1580173 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[113 (1501237435294547968)]} 0 1
   [junit4]   2> 1580179 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[114 (1501237435300839424)]} 0 1
   [junit4]   2> 1580183 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[114 (1501237435305033728)]} 0 1
   [junit4]   2> 1580187 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[115 (1501237435309228032)]} 0 1
   [junit4]   2> 1580191 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[115 (1501237435313422336)]} 0 1
   [junit4]   2> 1580195 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[116 (1501237435317616640)]} 0 1
   [junit4]   2> 1580203 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[116 (1501237435321810944)]} 0 4
   [junit4]   2> 1580210 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[117 (1501237435333345280)]} 0 1
   [junit4]   2> 1580214 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[117 (1501237435337539584)]} 0 1
   [junit4]   2> 1580217 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[118 (1501237435340685312)]} 0 1
   [junit4]   2> 1580222 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[118 (1501237435344879616)]} 0 1
   [junit4]   2> 1580231 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[119 (1501237435355365376)]} 0 1
   [junit4]   2> 1580250 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[119 (1501237435374239744)]} 0 2
   [junit4]   2> 1580264 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[120 (1501237435389968384)]} 0 1
   [junit4]   2> 1580282 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[120 (1501237435408842752)]} 0 1
   [junit4]   2> 1580292 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[121 (1501237435418279936)]} 0 2
   [junit4]   2> 1580305 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[121 (1501237435432960000)]} 0 1
   [junit4]   2> 1580311 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[122 (1501237435438202880)]} 0 2
   [junit4]   2> 1580321 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[122 (1501237435449737216)]} 0 1
   [junit4]   2> 1580329 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[123 (1501237435456028672)]} 0 3
   [junit4]   2> 1580334 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[123 (1501237435463368704)]} 0 1
   [junit4]   2> 1580339 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[124 (1501237435467563008)]} 0 2
   [junit4]   2> 1580344 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[124 (1501237435473854464)]} 0 2
   [junit4]   2> 1580348 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[125 (1501237435478048768)]} 0 1
   [junit4]   2> 1580353 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[125 (1501237435483291648)]} 0 1
   [junit4]   2> 1580356 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[126 (1501237435487485952)]} 0 0
   [junit4]   2> 1580361 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[126 (1501237435491680256)]} 0 1
   [junit4]   2> 1580365 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[127 (1501237435495874560)]} 0 1
   [junit4]   2> 1580369 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[127 (1501237435500068864)]} 0 1
   [junit4]   2> 1580373 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[128 (1501237435504263168)]} 0 1
   [junit4]   2> 1580377 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[128 (1501237435508457472)]} 0 1
   [junit4]   2> 1580381 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[129 (1501237435512651776)]} 0 1
   [junit4]   2> 1580384 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[129 (1501237435516846080)]} 0 0
   [junit4]   2> 1580388 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[130 (1501237435519991808)]} 0 1
   [junit4]   2> 1580392 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[130 (1501237435524186112)]} 0 1
   [junit4]   2> 1580396 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[131 (1501237435528380416)]} 0 1
   [junit4]   2> 1580399 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[131 (1501237435531526144)]} 0 1
   [junit4]   2> 1580403 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[132 (1501237435535720448)]} 0 1
   [junit4]   2> 1580407 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[132 (1501237435539914752)]} 0 1
   [junit4]   2> 1580412 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[133 (1501237435545157632)]} 0 1
   [junit4]   2> 1580416 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[133 (1501237435549351936)]} 0 1
   [junit4]   2> 1580420 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[134 (1501237435553546240)]} 0 1
   [junit4]   2> 1580424 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[134 (1501237435557740544)]} 0 1
   [junit4]   2> 1580428 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[135 (1501237435561934848)]} 0 1
   [junit4]   2> 1580432 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[135 (1501237435566129152)]} 0 1
   [junit4]   2> 1580436 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[136 (1501237435570323456)]} 0 1
   [junit4]   2> 1580441 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[136 (1501237435574517760)]} 0 2
   [junit4]   2> 1580451 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[137 (1501237435586052096)]} 0 2
   [junit4]   2> 1580455 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[137 (1501237435590246400)]} 0 1
   [junit4]   2> 1580464 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[138 (1501237435599683584)]} 0 1
   [junit4]   2> 1580470 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[138 (1501237435605975040)]} 0 2
   [junit4]   2> 1580474 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[139 (1501237435610169344)]} 0 1
   [junit4]   2> 1580478 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[139 (1501237435614363648)]} 0 1
   [junit4]   2> 1580482 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[140 (1501237435618557952)]} 0 1
   [junit4]   2> 1580486 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[140 (1501237435622752256)]} 0 1
   [junit4]   2> 1580490 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[141 (1501237435626946560)]} 0 1
   [junit4]   2> 1580494 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[141 (1501237435631140864)]} 0 1
   [junit4]   2> 1580498 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[142 (1501237435635335168)]} 0 1
   [junit4]   2> 1580501 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[142 (1501237435638480896)]} 0 1
   [junit4]   2> 1580509 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[143 (1501237435646869504)]} 0 1
   [junit4]   2> 1580512 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[143 (1501237435650015232)]} 0 1
   [junit4]   2> 1580517 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[144 (1501237435655258112)]} 0 1
   [junit4]   2> 1580521 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[144 (1501237435659452416)]} 0 1
   [junit4]   2> 1580525 T8140 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[145 (1501237435663646720)]} 0 1
   [junit4]   2> 1580529 T8154 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[145 (1501237435667841024)]} 0 1
   [junit4]   2> 1580532 T8135 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[146 (1501237435672035328)]} 0 0
   [junit4]   2> 1580537 T8149 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[146 (1501237435675181056)]} 0 2
   [junit4]   2> 1580540 T8136 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[147 (1501237435680423936)]} 0 0
   [junit4]   2> 1580544 T8150 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[147 (1501237435683569664)]} 0 1
   [junit4]   2> 1580550 T8137 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[148 (1501237435687763968)]} 0 3
   [junit4]   2> 1580553 T8151 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[148 (1501237435693006848)]} 0 1
   [junit4]   2> 1580557 T8138 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[149 (1501237435698249728)]} 0 1
   [junit4]   2> 1580561 T8152 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[149 (1501237435701395456)]} 0 1
   [junit4]   2> 1580564 T8139 x:collection1 C1179 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1580573 T8139 x:collection1 C1179 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@7a73cfbe lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@24400869),segFN=segments_3,generation=3}
   [junit4]   2> 1580573 T8139 x:collection1 C1179 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 1580574 T8139 x:collection1 C1179 oass.SolrIndexSearcher.<init> Opening Searcher@29232af6[collection1] main
   [junit4]   2> 1580574 T8139 x:collection1 C1179 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1580576 T8144 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@29232af6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c9) Uninverting(_1(6.0.0):c59)))}
   [junit4]   2> 1580578 T8139 x:collection1 C1179 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 14
   [junit4]   2> 1580595 T8153 x:collection1 C1180 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1580628 T8153 x:collection1 C1180 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2a9609dc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@d5ab069),segFN=segments_3,generation=3}
   [junit4]   2> 1580629 T8153 x:collection1 C1180 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 1580629 T8153 x:collection1 C1180 oass.SolrIndexSearcher.<init> Opening Searcher@37b4af64[collection1] main
   [junit4]   2> 1580629 T8153 x:collection1 C1180 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 1580631 T8158 x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@37b4af64[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):c9) Uninverting(_1(6.0.0):c59)))}
   [junit4]   2> 1580631 T8153 x:collection1 C1180 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 37
   [junit4]   2> 1580644 T8140 x:collection1 C1179 oasc.SolrException.log Ignoring exception matching parameter cannot be negative
   [junit4]   2> 1580644 T8140 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*&distrib=false&start=-1&wt=javabin&version=2} status=400 QTime=1 
   [junit4]   2> 1580648 T8135 x:collection1 C1179 oasc.SolrException.log Ignoring exception matching parameter cannot be negative
   [junit4]   2> 1580649 T8135 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*&distrib=false&start=0&rows=-1&wt=javabin&version=2} status=400 QTime=1 
   [junit4]   2> 1580661 T8136 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=7 
   [junit4]   2> 1580691 T8149 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::213]:33332/|[ff01::114]:33332/&rows=10&version=2&q=*:*&NOW=1431691585373&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580708 T8150 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585373&ids=121,143,111,101,124,102,148,115,105,120&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::213]:33332/|[ff01::114]:33332/&wt=javabin&version=2} status=0 QTime=4 
   [junit4]   2> 1580719 T8154 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::213]:33332/|[ff01::114]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=42 
   [junit4]   2> 1580737 T8137 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580744 T8152 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=http://127.0.0.1:62188//collection1&rows=10&version=2&q=*:*&NOW=1431691585436&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580754 T8153 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585436&ids=100,13,147,126,127,1001,139,9,109,131&isShard=true&shard.url=http://127.0.0.1:62188//collection1&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 1580765 T8151 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=http://127.0.0.1:62188//collection1&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=25 
   [junit4]   2> 1580775 T8138 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=1 
   [junit4]   2> 1580793 T8150 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&rows=10&version=2&q=*:*&NOW=1431691585485&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580797 T8154 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585485&ids=11,2,123,112,113,103,149,138,109,140&isShard=true&shard.url=[ff01::114]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1580812 T8149 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=23 
   [junit4]   2> 1580815 T8139 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_f1+asc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580828 T8151 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+asc&shard.url=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::114]:33332/&rows=10&version=2&q=*:*&NOW=1431691585519&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580831 T8153 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585519&ids=132,102,125,115,127,106,1001,10,142,120&isShard=true&shard.url=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::114]:33332/&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1580833 T8152 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1|[ff01::114]:33332/&sort=n_f1+asc&wt=javabin&version=2} hits=68 status=0 QTime=10 
   [junit4]   2> 1580838 T8140 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_tf1+desc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580847 T8154 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_tf1+desc&shard.url=http://127.0.0.1:62188//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1431691585540&isShard=true&wt=javabin} hits=68 status=0 QTime=0 
   [junit4]   2> 1580851 T8149 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585540&ids=124,136,6,105,138,139,129,141,10,142&isShard=true&shard.url=http://127.0.0.1:62188//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 1580852 T8150 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=http://127.0.0.1:62188//collection1|[ff01::213]:33332/&sort=n_tf1+desc&wt=javabin&version=2} hits=68 status=0 QTime=8 
   [junit4]   2> 1580855 T8135 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_tf1+asc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580860 T8153 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_tf1+asc&shard.url=http://127.0.0.1:62188//collection1&rows=10&version=2&q=*:*&NOW=1431691585553&isShard=true&wt=javabin} hits=68 status=0 QTime=0 
   [junit4]   2> 1580868 T8152 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585553&ids=12,111,1,133,100,4,103,104,140,131&isShard=true&shard.url=http://127.0.0.1:62188//collection1&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 1580878 T8151 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=http://127.0.0.1:62188//collection1&sort=n_tf1+asc&wt=javabin&version=2} hits=68 status=0 QTime=21 
   [junit4]   2> 1580886 T8136 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_d1+desc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580895 T8149 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_d1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&rows=10&version=2&q=*:*&NOW=1431691585588&isShard=true&wt=javabin} hits=68 status=0 QTime=0 
   [junit4]   2> 1580901 T8150 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585588&ids=133,123,146,102,6,138,108,130,120,10&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&wt=javabin&version=2} status=0 QTime=1 
   [junit4]   2> 1580903 T8154 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:62188//collection1|[ff01::083]:33332/&sort=n_d1+desc&wt=javabin&version=2} hits=68 status=0 QTime=11 
   [junit4]   2> 1580906 T8137 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_d1+asc&wt=javabin&version=2} hits=68 status=0 QTime=0 
   [junit4]   2> 1580914 T8152 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_d1+asc&shard.url=[ff01::114]:33332/|[ff01::083]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1&rows=10&version=2&q=*:*&NOW=1431691585607&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580923 T8151 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585607&ids=144,122,101,134,125,5,127,1001,128,109&isShard=true&shard.url=[ff01::114]:33332/|[ff01::083]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1580925 T8153 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|[ff01::083]:33332/|[ff01::213]:33332/|http://127.0.0.1:62188//collection1&sort=n_d1+asc&wt=javabin&version=2} hits=68 status=0 QTime=14 
   [junit4]   2> 1580932 T8138 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_td1+desc&wt=javabin&version=2} hits=68 status=0 QTime=1 
   [junit4]   2> 1580941 T8150 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_td1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:62188//collection1&rows=10&version=2&q=*:*&NOW=1431691585633&isShard=true&wt=javabin} hits=68 status=0 QTime=1 
   [junit4]   2> 1580947 T8154 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585633&ids=122,102,135,146,4,1001,9,109,131,10&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:62188//collection1&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1580948 T8149 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:62188//collection1&sort=n_td1+desc&wt=javabin&version=2} hits=68 status=0 QTime=11 
   [junit4]   2> 1580952 T8139 x:collection1 C1179 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_td1+asc&wt=javabin&version=2} hits=68 status=0 QTime=1 
   [junit4]   2> 1580958 T8153 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_td1+asc&shard.url=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1&rows=10&version=2&q=*:*&NOW=1431691585652&isShard=true&wt=javabin} hits=68 status=0 QTime=0 
   [junit4]   2> 1580971 T8151 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&distrib=false&shards.purpose=64&NOW=1431691585652&ids=143,132,144,101,14,103,104,148,6,139&isShard=true&shard.url=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1&wt=javabin&version=2} status=0 QTime=0 
   [junit4]   2> 1580984 T8152 x:collection1 C1180 oasc.SolrCore.execute [collection1] webapp= path=/select params={q=*:*&shards=[ff01::213]:33332/|[ff01::083]:33332/|http://127.0.0.1:62188//collection1&sort=n_td1+asc&wt=javabin&

[...truncated too long message...]

ient$RemoteSolrException: Error from server at http://127.0.0.1:62194//collection1: java.lang.NullPointerException
   [junit4]    > 	at org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:102)
   [junit4]    > 	at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:744)
   [junit4]    > 	at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:727)
   [junit4]    > 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:388)
   [junit4]    > 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
   [junit4]    > 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2047)
   [junit4]    > 	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:636)
   [junit4]    > 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:436)
   [junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:198)
   [junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:178)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:105)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    > 	at org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
   [junit4]    > 	at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:364)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
   [junit4]    > 	at org.eclipse.jetty.server.Server.handle(Server.java:497)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
   [junit4]    > 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
   [junit4]    > 	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([47AE2BD9E5661547:CFFA14034B9A78BF]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:943)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:958)
   [junit4]    > 	at org.apache.solr.TestDistributedSearch.queryRandomUpServer(TestDistributedSearch.java:1133)
   [junit4]    > 	at org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1095)
   [junit4]    > 	at org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:968)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:982)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:935)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1698578 T8131 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.TestDistributedSearch 47AE2BD9E5661547-001
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, maxDocsPerChunk=367, blockSize=842), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, blockSize=842)), sim=DefaultSimilarity, locale=mk_MK, timezone=America/Miquelon
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=1,free=73843568,total=316669952
   [junit4]   2> NOTE: All tests run in this JVM: [TestEmbeddedSolrServerConstructors, ZkSolrClientTest, TestCodecSupport, IgnoreCommitOptimizeUpdateProcessorFactoryTest, NotRequiredUniqueKeyTest, DistributedFacetPivotWhiteBoxTest, TestFastOutputStream, CurrencyFieldXmlFileTest, OverriddenZkACLAndCredentialsProvidersTest, DeleteInactiveReplicaTest, SolrXmlInZkTest, TestReplicaProperties, TestFieldSortValues, DirectUpdateHandlerOptimizeTest, BasicDistributedZk2Test, SuggesterTSTTest, CoreAdminCreateDiscoverTest, LeaderElectionIntegrationTest, AtomicUpdatesTest, TestLRUStatsCache, AssignTest, IndexSchemaTest, TestMiniSolrCloudCluster, UpdateRequestProcessorFactoryTest, TestSolrJ, TestUtils, TestManagedSchemaFieldResource, ConvertedLegacyTest, TestShardHandlerFactory, SpellingQueryConverterTest, TestRecovery, LeaderFailoverAfterPartitionTest, TestLMJelinekMercerSimilarityFactory, RequestLoggingTest, DistributedFacetPivotSmallTest, FileBasedSpellCheckerTest, AddBlockUpdateTest, TestSolr4Spatial, TestDFRSimilarityFactory, CurrencyFieldOpenExchangeTest, TestZkChroot, BJQParserTest, DistributedQueueTest, AnalysisErrorHandlingTest, TestSolrDeletionPolicy2, DisMaxRequestHandlerTest, TestCryptoKeys, DistributedMLTComponentTest, PolyFieldTest, TestDefaultSimilarityFactory, TestConfigOverlay, FullSolrCloudDistribCmdsTest, TestFuzzyAnalyzedSuggestions, TestComplexPhraseQParserPlugin, CoreMergeIndexesAdminHandlerTest, PluginInfoTest, TestNamedUpdateProcessors, TestSchemaManager, TestSolrConfigHandler, BufferStoreTest, MultiThreadedOCPTest, DebugComponentTest, TestManagedStopFilterFactory, TestFastWriter, AddSchemaFieldsUpdateProcessorFactoryTest, SpatialRPTFieldTypeTest, CachingDirectoryFactoryTest, CopyFieldTest, TestAddFieldRealTimeGet, TestManagedSchemaDynamicFieldResource, TestValueSourceCache, TestManagedResource, ReturnFieldsTest, TestDefaultSearchFieldResource, DocValuesMissingTest, DistribDocExpirationUpdateProcessorTest, DeleteShardTest, IndexBasedSpellCheckerTest, CloudExitableDirectoryReaderTest, JsonLoaderTest, TestNoOpRegenerator, TestRemoteStreaming, TestReload, TestRTGBase, VMParamsZkACLAndCredentialsProvidersTest, HdfsCollectionsAPIDistributedZkTest, TestSolrDynamicMBean, ExternalFileFieldSortTest, SharedFSAutoReplicaFailoverTest, AutoCommitTest, MBeansHandlerTest, TestSearchPerf, StatelessScriptUpdateProcessorFactoryTest, MoreLikeThisHandlerTest, HdfsBasicDistributedZkTest, SolrRequestParserTest, BlockDirectoryTest, DeleteReplicaTest, HighlighterConfigTest, SpellPossibilityIteratorTest, RAMDirectoryFactoryTest, DistanceUnitsTest, TestIntervalFaceting, TestShortCircuitedRequests, HdfsThreadLeakTest, TestPHPSerializedResponseWriter, TestRealTimeGet, TestLMDirichletSimilarityFactory, TestCollationFieldDocValues, TestCoreDiscovery, EchoParamsTest, UUIDUpdateProcessorFallbackTest, ZkNodePropsTest, TestFoldingMultitermQuery, BlockCacheTest, SpatialFilterTest, CSVRequestHandlerTest, StandardRequestHandlerTest, TestStressUserVersions, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestRandomFaceting, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch]
   [junit4] Completed [259/490] on J1 in 120.67s, 1 test, 1 error <<< FAILURES!

[...truncated 723 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:526: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:474: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:229: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:512: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1415: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:973: There were test failures: 490 suites, 1958 tests, 1 error, 62 ignored (24 assumptions)

Total time: 68 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.8.0) - Build # 2308 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/2308/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=1691, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1691, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56658: Could not find collection : awholynewstresscollection_collection1_0
	at __randomizedtesting.SeedInfo.seed([EE5BB0C894289B08]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)




Build Log:
[...truncated 9348 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/init-core-data-001
   [junit4]   2> 192954 T1437 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 192954 T1437 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 192961 T1437 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 192962 T1438 oasc.ZkTestServer$2$1.setClientPort client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 192962 T1438 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 193063 T1437 oasc.ZkTestServer.run start zk server on port:56638
   [junit4]   2> 193064 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 193065 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 193076 T1445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37ab4172 name:ZooKeeperConnection Watcher:127.0.0.1:56638 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 193076 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 193077 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 193077 T1437 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 193083 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 193085 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 193088 T1448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dc4ddb2 name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 193088 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 193089 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 193089 T1437 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 193095 T1437 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 193100 T1437 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 193104 T1437 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 193108 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 193109 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 193115 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 193116 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 193123 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 193124 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 193128 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 193128 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 193132 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 193133 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 193137 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 193138 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 193143 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 193143 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 193148 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 193148 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 193152 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 193153 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 193157 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 193158 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 193162 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 193162 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 193170 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 193174 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 193178 T1451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51f450a2 name:ZooKeeperConnection Watcher:127.0.0.1:56638 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 193178 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 193179 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 193179 T1437 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 193184 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 193186 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 193190 T1454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69c15046 name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 193191 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 193191 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 193192 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
   [junit4]   2> 193192 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
   [junit4]   2> 193199 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
   [junit4]   2> 193200 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
   [junit4]   2> 193205 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
   [junit4]   2> 193206 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/enumsConfig.xml
   [junit4]   2> 193211 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 193212 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 193217 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
   [junit4]   2> 193217 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
   [junit4]   2> 193222 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
   [junit4]   2> 193222 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
   [junit4]   2> 193229 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
   [junit4]   2> 193229 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
   [junit4]   2> 193233 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
   [junit4]   2> 193234 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
   [junit4]   2> 193238 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 193239 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
   [junit4]   2> 193243 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
   [junit4]   2> 193244 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
   [junit4]   2> 193249 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
   [junit4]   2> 193251 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
   [junit4]   2> 193271 T1437 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
   [junit4]   2> 193271 T1437 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
   [junit4]   2> 193867 T1437 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1
   [junit4]   2> 193871 T1437 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 193877 T1437 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@4077e3f6{/,null,AVAILABLE}
   [junit4]   2> 193957 T1437 oejs.AbstractConnector.doStart Started ServerConnector@76da0a8d{HTTP/1.1}{127.0.0.1:56649}
   [junit4]   2> 193958 T1437 oejs.Server.doStart Started @198321ms
   [junit4]   2> 193958 T1437 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {hostContext=/, hostPort=56649, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores}
   [junit4]   2> 193959 T1437 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@74a14482
   [junit4]   2> 193959 T1437 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/'
   [junit4]   2> 194005 T1437 oasc.SolrXmlConfig.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/solr.xml
   [junit4]   2> 194018 T1437 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores
   [junit4]   2> 194018 T1437 oasc.CoreContainer.<init> New CoreContainer 1448277674
   [junit4]   2> 194019 T1437 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/]
   [junit4]   2> 194019 T1437 oasc.CoreContainer.load loading shared library: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/lib
   [junit4]   2> 194020 T1437 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/lib).
   [junit4]   2> 194037 T1437 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 194042 T1437 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 194043 T1437 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 194043 T1437 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 194043 T1437 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 194044 T1437 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56638/solr
   [junit4]   2> 194044 T1437 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 194044 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 194047 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 194051 T1468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c216f73 name:ZooKeeperConnection Watcher:127.0.0.1:56638 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 194052 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 194053 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 194065 T1437 n:127.0.0.1:56649_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 194069 T1471 n:127.0.0.1:56649_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@524a25ed name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 194070 T1437 n:127.0.0.1:56649_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 194074 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 194080 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 194099 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 194105 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 194111 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 194118 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 194125 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 194130 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 194133 T1437 n:127.0.0.1:56649_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56649_
   [junit4]   2> 194135 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56649_
   [junit4]   2> 194141 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 194150 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 194156 T1437 n:127.0.0.1:56649_ oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 194161 T1437 n:127.0.0.1:56649_ oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56649_
   [junit4]   2> 194162 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 194167 T1437 n:127.0.0.1:56649_ oasc.Overseer.start Overseer (id=93828744184070149-127.0.0.1:56649_-n_0000000000) starting
   [junit4]   2> 194173 T1437 n:127.0.0.1:56649_ oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 194196 T1437 n:127.0.0.1:56649_ oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 194197 T1473 n:127.0.0.1:56649_ oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 194197 T1437 n:127.0.0.1:56649_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 194199 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 194212 T1437 n:127.0.0.1:56649_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 194220 T1437 n:127.0.0.1:56649_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores
   [junit4]   2> 194222 T1437 n:127.0.0.1:56649_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1, collection=control_collection, absoluteInstDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 194222 T1437 n:127.0.0.1:56649_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/
   [junit4]   2> 194223 T1437 n:127.0.0.1:56649_ oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 194225 T1475 n:127.0.0.1:56649_ c:control_collection x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 194225 T1475 n:127.0.0.1:56649_ c:control_collection x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 194227 T1475 n:127.0.0.1:56649_ oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 194227 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 194230 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56649",
   [junit4]   2> 	  "node_name":"127.0.0.1:56649_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 194232 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56649",
   [junit4]   2> 	  "node_name":"127.0.0.1:56649_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 194232 T1472 n:127.0.0.1:56649_ oasco.ClusterStateMutator.createCollection building a new cName: control_collection
   [junit4]   2> 194232 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 194235 T1471 n:127.0.0.1:56649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 195229 T1475 n:127.0.0.1:56649_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 195229 T1475 n:127.0.0.1:56649_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 195230 T1475 n:127.0.0.1:56649_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 195230 T1475 n:127.0.0.1:56649_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 195232 T1475 n:127.0.0.1:56649_ oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 195232 T1475 n:127.0.0.1:56649_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/'
   [junit4]   2> 195250 T1475 n:127.0.0.1:56649_ oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 195270 T1475 n:127.0.0.1:56649_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 195277 T1475 n:127.0.0.1:56649_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 195298 T1475 n:127.0.0.1:56649_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 195300 T1475 n:127.0.0.1:56649_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 195305 T1475 n:127.0.0.1:56649_ oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 195569 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 195586 T1475 n:127.0.0.1:56649_ oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 195587 T1475 n:127.0.0.1:56649_ oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 195661 T1475 n:127.0.0.1:56649_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 195676 T1475 n:127.0.0.1:56649_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 195685 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 195688 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 195688 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 195689 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 195701 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 195701 T1475 n:127.0.0.1:56649_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 195701 T1475 n:127.0.0.1:56649_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 195701 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 195702 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 195702 T1475 n:127.0.0.1:56649_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a4f72d
   [junit4]   2> 195703 T1475 n:127.0.0.1:56649_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/data
   [junit4]   2> 195703 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/data/index/
   [junit4]   2> 195703 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 195703 T1475 n:127.0.0.1:56649_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/control-001/cores/collection1/data/index
   [junit4]   2> 195703 T1475 n:127.0.0.1:56649_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 195724 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@c9bc894 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d4d52c4),segFN=segments_1,generation=1}
   [junit4]   2> 195724 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 195745 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 195746 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 195746 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 195746 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 195746 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 195747 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 195747 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 195755 T1475 n:127.0.0.1:56649_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 195756 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 195761 T1475 n:127.0.0.1:56649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 195763 T1475 n:127.0.0.1:56649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 195765 T1475 n:127.0.0.1:56649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 195767 T1475 n:127.0.0.1:56649_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 195771 T1475 n:127.0.0.1:56649_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 195772 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 195774 T1475 n:127.0.0.1:56649_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 195774 T1475 n:127.0.0.1:56649_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 195776 T1475 n:127.0.0.1:56649_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 195776 T1475 n:127.0.0.1:56649_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 195776 T1475 n:127.0.0.1:56649_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=86.2900390625, floorSegmentMB=0.8701171875, forceMergeDeletesPctAllowed=4.6701660860091145, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 195777 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@c9bc894 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d4d52c4),segFN=segments_1,generation=1}
   [junit4]   2> 195777 T1475 n:127.0.0.1:56649_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 195778 T1475 n:127.0.0.1:56649_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@49711ad8[collection1] main
   [junit4]   2> 195778 T1475 n:127.0.0.1:56649_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 195786 T1475 n:127.0.0.1:56649_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 195787 T1475 n:127.0.0.1:56649_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 195788 T1475 n:127.0.0.1:56649_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 195788 T1475 n:127.0.0.1:56649_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 195788 T1475 n:127.0.0.1:56649_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 195791 T1475 n:127.0.0.1:56649_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 195792 T1475 n:127.0.0.1:56649_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 195792 T1475 n:127.0.0.1:56649_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 195792 T1475 n:127.0.0.1:56649_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 195800 T1476 n:127.0.0.1:56649_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@49711ad8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 195800 T1475 n:127.0.0.1:56649_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 195804 T1475 n:127.0.0.1:56649_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 195808 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56649 collection:control_collection shard:shard1
   [junit4]   2> 195810 T1437 n:127.0.0.1:56649_ oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1
   [junit4]   2> 195811 T1437 n:127.0.0.1:56649_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 195812 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 195814 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 195818 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 195820 T1482 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@742d42ff name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 195820 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 195821 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 195821 T1437 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 195829 T1437 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 195830 T1437 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 195830 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 195832 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 195837 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 195837 T1485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e5ef5a9 name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 195837 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 195838 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 195845 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 195849 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "name":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "stateFormat":"2"} current state version: 1
   [junit4]   2> 195850 T1472 n:127.0.0.1:56649_ oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 195852 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 195852 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C572 name=collection1 org.apache.solr.core.SolrCore@7be6ab7e url=http://127.0.0.1:56649/collection1 node=127.0.0.1:56649_ C572_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:56649, node_name=127.0.0.1:56649_, state=down}
   [junit4]   2> 195852 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 C572 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56649/collection1/
   [junit4]   2> 195852 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 C572 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 195852 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 C572 oasc.SyncStrategy.syncToMe http://127.0.0.1:56649/collection1/ has no replicas
   [junit4]   2> 195853 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56649/collection1/ shard1
   [junit4]   2> 195853 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 195861 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
   [junit4]   2> 195865 T1482 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 195867 T1471 n:127.0.0.1:56649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 195894 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"} current state version: 2
   [junit4]   2> 195910 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 195913 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56649",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 2
   [junit4]   2> 196027 T1471 n:127.0.0.1:56649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 196033 T1482 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 196095 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register We are http://127.0.0.1:56649/collection1/ and leader is http://127.0.0.1:56649/collection1/
   [junit4]   2> 196096 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56649
   [junit4]   2> 196096 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 196096 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 196096 T1479 n:127.0.0.1:56649_ c:control_collection s:shard1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 196101 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 196111 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56649",
   [junit4]   2> 	  "node_name":"127.0.0.1:56649_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 196120 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56649",
   [junit4]   2> 	  "node_name":"127.0.0.1:56649_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 196252 T1482 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 196254 T1471 n:127.0.0.1:56649_ oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 196824 T1437 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1
   [junit4]   2> 196827 T1437 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001
   [junit4]   2> 196839 T1437 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 196855 T1437 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@4c8b583e{/,null,AVAILABLE}
   [junit4]   2> 196857 T1437 oejs.AbstractConnector.doStart Started ServerConnector@2de29b5b{HTTP/1.1}{127.0.0.1:56654}
   [junit4]   2> 196857 T1437 oejs.Server.doStart Started @201219ms
   [junit4]   2> 196858 T1437 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=56654, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores}
   [junit4]   2> 196858 T1437 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@74a14482
   [junit4]   2> 196858 T1437 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/'
   [junit4]   2> 196933 T1437 oasc.SolrXmlConfig.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/solr.xml
   [junit4]   2> 196958 T1437 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores
   [junit4]   2> 196958 T1437 oasc.CoreContainer.<init> New CoreContainer 1118073245
   [junit4]   2> 196958 T1437 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/]
   [junit4]   2> 196959 T1437 oasc.CoreContainer.load loading shared library: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/lib
   [junit4]   2> 196960 T1437 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/lib).
   [junit4]   2> 196994 T1437 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 197004 T1437 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 197006 T1437 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 197006 T1437 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 197006 T1437 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 197007 T1437 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56638/solr
   [junit4]   2> 197007 T1437 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 197007 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 197015 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 197029 T1499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5eb65f7b name:ZooKeeperConnection Watcher:127.0.0.1:56638 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 197031 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 197032 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 197052 T1437 n:127.0.0.1:56654_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 197064 T1502 n:127.0.0.1:56654_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34990912 name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 197067 T1437 n:127.0.0.1:56654_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 197105 T1437 n:127.0.0.1:56654_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 198144 T1437 n:127.0.0.1:56654_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56654_
   [junit4]   2> 198146 T1437 n:127.0.0.1:56654_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56654_
   [junit4]   2> 198159 T1437 n:127.0.0.1:56654_ oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 198163 T1437 n:127.0.0.1:56654_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 198165 T1437 n:127.0.0.1:56654_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores
   [junit4]   2> 198166 T1437 n:127.0.0.1:56654_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1, collection=collection1, absoluteInstDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 198167 T1437 n:127.0.0.1:56654_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/
   [junit4]   2> 198167 T1437 n:127.0.0.1:56654_ oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 198174 T1503 n:127.0.0.1:56654_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 198174 T1503 n:127.0.0.1:56654_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 198178 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 198180 T1503 n:127.0.0.1:56654_ oasc.ZkController.preRegister Registering watch for external collection collection1
   [junit4]   2> 198180 T1503 n:127.0.0.1:56654_ oascc.ZkStateReader.addZkWatch addZkWatch collection1
   [junit4]   2> 198181 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56654",
   [junit4]   2> 	  "node_name":"127.0.0.1:56654_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 4
   [junit4]   2> 198182 T1503 n:127.0.0.1:56654_ oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 198186 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56654",
   [junit4]   2> 	  "node_name":"127.0.0.1:56654_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 198187 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 198187 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 198195 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 0
   [junit4]   2> 198197 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 198199 T1503 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1 
   [junit4]   2> 198199 T1503 n:127.0.0.1:56654_ oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 198199 T1503 n:127.0.0.1:56654_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 198199 T1503 n:127.0.0.1:56654_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 198201 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 1 
   [junit4]   2> 198204 T1503 n:127.0.0.1:56654_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 198204 T1503 n:127.0.0.1:56654_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 198205 T1503 n:127.0.0.1:56654_ oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 198206 T1503 n:127.0.0.1:56654_ oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/'
   [junit4]   2> 198228 T1503 n:127.0.0.1:56654_ oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 198239 T1503 n:127.0.0.1:56654_ oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 198250 T1503 n:127.0.0.1:56654_ oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 198271 T1503 n:127.0.0.1:56654_ oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 198274 T1503 n:127.0.0.1:56654_ oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 198279 T1503 n:127.0.0.1:56654_ oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 198590 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 198601 T1503 n:127.0.0.1:56654_ oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 198602 T1503 n:127.0.0.1:56654_ oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 198655 T1503 n:127.0.0.1:56654_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 198659 T1503 n:127.0.0.1:56654_ oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 198677 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 198685 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 198685 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 198685 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 198691 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 198691 T1503 n:127.0.0.1:56654_ oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 198691 T1503 n:127.0.0.1:56654_ oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 198692 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 198692 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.<init> [[collection1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/], dataDir=[null]
   [junit4]   2> 198692 T1503 n:127.0.0.1:56654_ x:collection1 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1a4f72d
   [junit4]   2> 198693 T1503 n:127.0.0.1:56654_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/data
   [junit4]   2> 198693 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/data/index/
   [junit4]   2> 198693 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 198693 T1503 n:127.0.0.1:56654_ x:collection1 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-1-001/cores/collection1/data/index
   [junit4]   2> 198694 T1503 n:127.0.0.1:56654_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=38, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 198700 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6788e564 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54bae77e),segFN=segments_1,generation=1}
   [junit4]   2> 198700 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 198711 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 198711 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 198711 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 198712 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 198718 T1503 n:127.0.0.1:56654_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 198721 T1503 n:127.0.0.1:56654_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 198723 T1503 n:127.0.0.1:56654_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 198724 T1503 n:127.0.0.1:56654_ x:collection1 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 198737 T1503 n:127.0.0.1:56654_ x:collection1 oasc.RequestHandlers.initHandlersFromConfig Registered paths: /admin/mbeans,standard,/update/csv,/update/json/docs,/admin/luke,/admin/segments,/get,/admin/system,/replication,/admin/properties,/config,/schema,/admin/plugins,/admin/logging,/update/json,/admin/threads,/admin/ping,/update,/admin/file
   [junit4]   2> 198738 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 198765 T1503 n:127.0.0.1:56654_ x:collection1 oasu.UpdateHandler.<init> Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 198765 T1503 n:127.0.0.1:56654_ x:collection1 oasu.UpdateLog.init Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10
   [junit4]   2> 198766 T1503 n:127.0.0.1:56654_ x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 198766 T1503 n:127.0.0.1:56654_ x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 198771 T1503 n:127.0.0.1:56654_ x:collection1 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=13, maxMergedSegmentMB=86.2900390625, floorSegmentMB=0.8701171875, forceMergeDeletesPctAllowed=4.6701660860091145, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 198772 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6788e564 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@54bae77e),segFN=segments_1,generation=1}
   [junit4]   2> 198772 T1503 n:127.0.0.1:56654_ x:collection1 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 198773 T1503 n:127.0.0.1:56654_ x:collection1 oass.SolrIndexSearcher.<init> Opening Searcher@4ee29d68[collection1] main
   [junit4]   2> 198773 T1503 n:127.0.0.1:56654_ x:collection1 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 198775 T1503 n:127.0.0.1:56654_ x:collection1 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 198775 T1503 n:127.0.0.1:56654_ x:collection1 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 198776 T1503 n:127.0.0.1:56654_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 198776 T1503 n:127.0.0.1:56654_ x:collection1 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 198777 T1503 n:127.0.0.1:56654_ x:collection1 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 198778 T1503 n:127.0.0.1:56654_ x:collection1 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 198778 T1503 n:127.0.0.1:56654_ x:collection1 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 198779 T1503 n:127.0.0.1:56654_ x:collection1 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 198779 T1503 n:127.0.0.1:56654_ x:collection1 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 198779 T1504 n:127.0.0.1:56654_ x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ee29d68[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 198780 T1503 n:127.0.0.1:56654_ x:collection1 oasc.ZkController.getConfDirListeners watch zkdir /configs/conf1
   [junit4]   2> 198783 T1503 n:127.0.0.1:56654_ x:collection1 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 198787 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56654 collection:collection1 shard:shard2
   [junit4]   2> 198789 T1437 n:127.0.0.1:56654_ oass.SolrDispatchFilter.init user.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1
   [junit4]   2> 198789 T1437 n:127.0.0.1:56654_ oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 198790 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 198838 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 198842 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 198844 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 198847 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 198847 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1"} current state version: 4
   [junit4]   2> ASYNC  NEW_CORE C573 name=collection1 org.apache.solr.core.SolrCore@5300ef10 url=http://127.0.0.1:56654/collection1 node=127.0.0.1:56654_ C573_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:56654, node_name=127.0.0.1:56654_, state=down}
   [junit4]   2> 198847 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 C573 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56654/collection1/
   [junit4]   2> 198848 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 C573 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 198848 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 C573 oasc.SyncStrategy.syncToMe http://127.0.0.1:56654/collection1/ has no replicas
   [junit4]   2> 198848 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56654/collection1/ shard2
   [junit4]   2> 198849 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 198851 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 1
   [junit4]   2> 198852 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 198854 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 2 
   [junit4]   2> 198865 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 198871 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56654",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 198875 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 2
   [junit4]   2> 198881 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 198886 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 3 
   [junit4]   2> 198927 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register We are http://127.0.0.1:56654/collection1/ and leader is http://127.0.0.1:56654/collection1/
   [junit4]   2> 198927 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56654
   [junit4]   2> 198928 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 198928 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 198928 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 198932 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 198939 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56654",
   [junit4]   2> 	  "node_name":"127.0.0.1:56654_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 4
   [junit4]   2> 198946 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56654",
   [junit4]   2> 	  "node_name":"127.0.0.1:56654_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 198958 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 3
   [junit4]   2> 198959 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 2)
   [junit4]   2> 198959 T1507 n:127.0.0.1:56654_ c:collection1 s:shard2 x:collection1 oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 
   [junit4]   2> 198961 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 
   [junit4]   2> 199385 T1437 oas.SolrTestCaseJ4.writeCoreProperties Writing core.properties file to /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores/collection1
   [junit4]   2> 199388 T1437 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001
   [junit4]   2> 199391 T1437 oejs.Server.doStart jetty-9.2.10.v20150310
   [junit4]   2> 199394 T1437 oejsh.ContextHandler.doStart Started o.e.j.s.ServletContextHandler@5fb5d584{/,null,AVAILABLE}
   [junit4]   2> 199395 T1437 oejs.AbstractConnector.doStart Started ServerConnector@14ebcee5{HTTP/1.1}{127.0.0.1:56658}
   [junit4]   2> 199396 T1437 oejs.Server.doStart Started @203756ms
   [junit4]   2> 199396 T1437 oascse.JettySolrRunner$1.lifeCycleStarted Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=56658, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores}
   [junit4]   2> 199396 T1437 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@74a14482
   [junit4]   2> 199396 T1437 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/'
   [junit4]   2> 199430 T1437 oasc.SolrXmlConfig.fromFile Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/solr.xml
   [junit4]   2> 199440 T1437 oasc.CorePropertiesLocator.<init> Config-defined core root directory: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores
   [junit4]   2> 199441 T1437 oasc.CoreContainer.<init> New CoreContainer 1324839829
   [junit4]   2> 199441 T1437 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/]
   [junit4]   2> 199441 T1437 oasc.CoreContainer.load loading shared library: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/lib
   [junit4]   2> 199442 T1437 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: lib (resolved as: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/lib).
   [junit4]   2> 199458 T1437 oashc.HttpShardHandlerFactory.init created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
   [junit4]   2> 199463 T1437 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 199463 T1437 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 199464 T1437 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 199465 T1437 oasc.CoreContainer.load Node Name: 127.0.0.1
   [junit4]   2> 199465 T1437 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56638/solr
   [junit4]   2> 199465 T1437 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 199465 T1437 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 199466 T1437 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 199469 T1521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@600447f name:ZooKeeperConnection Watcher:127.0.0.1:56638 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 199470 T1437 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 199470 T1437 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 199477 T1437 n:127.0.0.1:56658_ oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 199480 T1524 n:127.0.0.1:56658_ oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47aa745 name:ZooKeeperConnection Watcher:127.0.0.1:56638/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 199480 T1437 n:127.0.0.1:56658_ oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 199492 T1437 n:127.0.0.1:56658_ oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 200509 T1437 n:127.0.0.1:56658_ oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56658_
   [junit4]   2> 200510 T1437 n:127.0.0.1:56658_ oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56658_
   [junit4]   2> 200516 T1437 n:127.0.0.1:56658_ oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 200520 T1437 n:127.0.0.1:56658_ oasc.CoreContainer.intializeAuthorizationPlugin Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 200523 T1437 n:127.0.0.1:56658_ oasc.CorePropertiesLocator.discover Looking for core definitions underneath /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores
   [junit4]   2> 200524 T1437 n:127.0.0.1:56658_ oasc.CoreDescriptor.<init> CORE DESCRIPTOR: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, instanceDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores/collection1, collection=collection1, absoluteInstDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores/collection1/, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 200525 T1437 n:127.0.0.1:56658_ oasc.CorePropertiesLocator.discoverUnder Found core collection1 in /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest EE5BB0C894289B08-001/shard-2-001/cores/collection1/
   [junit4]   2> 200525 T1437 n:127.0.0.1:56658_ oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 200530 T1525 n:127.0.0.1:56658_ c:collection1 x:collection1 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 200530 T1525 n:127.0.0.1:56658_ c:collection1 x:collection1 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 200533 T1471 n:127.0.0.1:56649_ oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 200534 T1525 n:127.0.0.1:56658_ oasc.ZkController.preRegister Registering watch for external collection collection1
   [junit4]   2> 200534 T1525 n:127.0.0.1:56658_ oascc.ZkStateReader.addZkWatch addZkWatch collection1
   [junit4]   2> 200534 T1472 n:127.0.0.1:56649_ oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56658",
   [junit4]   2> 	  "node_name":"127.0.0.1:56658_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 4
   [junit4]   2> 200536 T1525 n:127.0.0.1:56658_ oascc.ZkStateReader.addZkWatch Updating collection state at /collections/collection1/state.json from ZooKeeper... 
   [junit4]   2> 200536 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56658",
   [junit4]   2> 	  "node_name":"127.0.0.1:56658_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 200536 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 200536 T1472 n:127.0.0.1:56649_ oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 200538 T1472 n:127.0.0.1:56649_ oasco.ZkStateWriter.writePendingUpdates going to update_collection /collections/collection1/state.json version: 4
   [junit4]   2> 200539 T1525 n:127.0.0.1:56658_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 4 
   [junit4]   2> 200539 T1525 n:127.0.0.1:56658_ oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 200539 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 3)
   [junit4]   2> 200540 T1524 n:127.0.0.1:56658_ oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json for collection collection1 has occurred - updating... (live nodes size: 3)
   [junit4]   2> 200541 T1502 n:127.0.0.1:56654_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 
   [junit4]   2> 200541 T1524 n:127.0.0.1:56658_ oascc.ZkStateReader.updateWatchedCollection Updating data for collection1 to ver 5 
   [junit4]   2> 201541 T1525 n:127.0.0.1:56658_ oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 201541 T1525 n:127.0.0.1:56658_ oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 201542 T1525 n:127.0.0.1:56658_ oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 201542 T1525 n:127.0.0.1:56658_ oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 201544 T1525 n:127.0.0.1:56658_ oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 201544 T1525 n:127.0.0.1:56658_ oasc.SolrResourceLoader.<init> new SolrResou

[...truncated too long message...]

ped o.e.j.s.ServletContextHandler@1e2d1510{/,null,UNAVAILABLE}
   [junit4]   2> 395300 T1437 c:collection1 s:shard2 x:collection1 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 56649
   [junit4]   2> 395304 T1437 c:collection1 s:shard2 x:collection1 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56638 56638
   [junit4]   2> 395333 T1438 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56638 56638
   [junit4]   2> 396262 T1438 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		16	/solr/aliases.json
   [junit4]   2> 		16	/solr/clusterstate.json
   [junit4]   2> 		7	/solr/configs/conf2
   [junit4]   2> 		6	/solr/configs/conf1
   [junit4]   2> 		5	/solr/collections/nodes_used_collection/state.json
   [junit4]   2> 		5	/solr/collections/halfdeletedcollection2/state.json
   [junit4]   2> 		5	/solr/collections/awholynewstresscollection_collection0_0/state.json
   [junit4]   2> 		5	/solr/collections/awholynewstresscollection_collection1_0/state.json
   [junit4]   2> 		4	/solr/collections/collection1/state.json
   [junit4]   2> 		3	/solr/collections/addReplicaColl/state.json
   [junit4]   2> 		2	/solr/collections/acollectionafterbaddelete/state.json
   [junit4]   2> 		2	/solr/collections/awholynewcollection_2/state.json
   [junit4]   2> 		2	/solr/collections/awholynewcollection_0/state.json
   [junit4]   2> 		2	/solr/collections/halfdeletedcollection/state.json
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent data watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		3	/solr/overseer_elect/election/93828744184070149-127.0.0.1:56649_-n_0000000000
   [junit4]   2> 		3	/solr/collections/awholynewstresscollection_collection1_0/leader_elect/shard2/election/93828744184070159-core_node16-n_0000000000
   [junit4]   2> 		3	/solr/collections/awholynewstresscollection_collection1_0/leader_elect/shard3/election/93828744184070159-core_node19-n_0000000000
   [junit4]   2> 		2	/solr/collections/awholynewstresscollection_collection0_0/leader_elect/shard3/election/93828744184070157-core_node21-n_0000000000
   [junit4]   2> 		2	/solr/collections/halfdeletedcollection2/leader_elect/shard1/election/93828744184070159-core_node1-n_0000000000
   [junit4]   2> 		2	/solr/collections/awholynewstresscollection_collection1_0/leader_elect/shard5/election/93828744184070159-core_node14-n_0000000000
   [junit4]   2> 		2	/solr/collections/halfdeletedcollection2/leader_elect/shard3/election/93828744184070157-core_node2-n_0000000000
   [junit4]   2> 		2	/solr/collections/addReplicaColl/leader_elect/shard2/election/93828744184070159-core_node1-n_0000000000
   [junit4]   2> 		2	/solr/collections/halfdeletedcollection2/leader_elect/shard2/election/93828744184070155-core_node4-n_0000000000
   [junit4]   2> 		2	/solr/collections/awholynewstresscollection_collection1_0/leader_elect/shard2/election/93828744184070155-core_node12-n_0000000001
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		14	/solr/live_nodes
   [junit4]   2> 		3	/solr/overseer/queue
   [junit4]   2> 		3	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=test -Dtests.seed=EE5BB0C894289B08 -Dtests.slow=true -Dtests.locale=zh_HK -Dtests.timezone=Africa/Freetown -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    203s J1 | CollectionsAPIDistributedZkTest.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1691, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56658: Could not find collection : awholynewstresscollection_collection1_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EE5BB0C894289B08]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1690, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56658: Could not find collection : awholynewstresscollection_collection0_0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EE5BB0C894289B08]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:560)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:235)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:227)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:376)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:328)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1086)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:856)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:799)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1220)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:884)
   [junit4]   2> 396284 T1437 c:collection1 s:shard2 x:collection1 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {other_tl1=PostingsFormat(name=MockRandom), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), _version_=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=PostingsFormat(name=MockRandom), a_i1=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=PostingsFormat(name=LuceneFixedGap), range_facet_l=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene50), range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct)}, sim=DefaultSimilarity, locale=zh_HK, timezone=Africa/Freetown
   [junit4]   2> NOTE: Mac OS X 10.8.5 x86_64/Oracle Corporation 1.8.0_45 (64-bit)/cpus=3,threads=1,free=143700960,total=256221184
   [junit4]   2> NOTE: All tests run in this JVM: [TestCryptoKeys, SolrRequestParserTest, TestInfoStreamLogging, TestJettySolrRunner, TestManagedSchema, PrimitiveFieldTypeTest, DistributedSuggestComponentTest, SignatureUpdateProcessorFactoryTest, HdfsDirectoryTest, DistribDocExpirationUpdateProcessorTest, SuggesterTSTTest, TestDynamicFieldCollectionResource, OpenExchangeRatesOrgProviderTest, TestReload, TestSolrDeletionPolicy1, TestTrie, IndexBasedSpellCheckerTest, TestCloudInspectUtil, SpellCheckCollatorTest, CollectionsAPIDistributedZkTest]
   [junit4] Completed [47/490] on J1 in 204.44s, 1 test, 1 error <<< FAILURES!

[...truncated 1392 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:526: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:474: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/build.xml:61: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/extra-targets.xml:39: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:229: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:512: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1415: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:973: There were test failures: 490 suites, 1958 tests, 1 error, 62 ignored (24 assumptions)

Total time: 73 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any