You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/07/10 04:36:39 UTC

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_60) - Build # 10772 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/10772/
Java: 32bit/jdk1.7.0_60 -server -XX:+UseConcMarkSweepGC

1 tests failed.
REGRESSION:  org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.testMultiCore

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
	at __randomizedtesting.SeedInfo.seed([3C116687DE339825:B8397D7411E09510]:0)
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
	at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:884)
	at org.eclipse.jetty.io.nio.SelectorManager.doStop(SelectorManager.java:330)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.util.component.AggregateLifeCycle.doStop(AggregateLifeCycle.java:107)
	at org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:354)
	at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStop(SslSelectChannelConnector.java:642)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.eclipse.jetty.server.Server.doStop(Server.java:338)
	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:457)
	at org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.tearDown(MultiCoreExampleJettyTest.java:63)
	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12543 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/init-core-data-001
   [junit4]   2> 42204 T88 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 42208 T88 oas.SolrTestCaseJ4.setUp ###Starting testMultiCore
   [junit4]   2> 42216 T88 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 42409 T88 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 42412 T88 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:36581
   [junit4]   2> 42415 T88 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 42415 T88 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 42415 T88 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore
   [junit4]   2> 42416 T88 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/'
   [junit4]   2> 42435 T88 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/solr.xml
   [junit4]   2> 42488 T88 oasc.CoreContainer.<init> New CoreContainer 16940592
   [junit4]   2> 42489 T88 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/]
   [junit4]   2> 42490 T88 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 42490 T88 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 42490 T88 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 42490 T88 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 42491 T88 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 42491 T88 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 42491 T88 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 42491 T88 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 42492 T88 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 42495 T88 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 42499 T88 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 42500 T88 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 42509 T98 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/'
   [junit4]   2> 42509 T99 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/'
   [junit4]   2> 42541 T98 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 42570 T99 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 42583 T98 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 42583 T98 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 42588 T98 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 42596 T98 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 42596 T98 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 42597 T98 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 42598 T98 oasc.CoreContainer.create Creating SolrCore 'core0' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/
   [junit4]   2> 42598 T98 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 42606 T99 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 42607 T99 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 42611 T98 oasc.SolrCore.<init> [core0] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/
   [junit4]   2> 42611 T98 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 42611 T99 oass.IndexSchema.readSchema [core1] Schema name=example core one
   [junit4]   2> 42619 T99 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 42619 T99 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 42620 T99 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 42620 T98 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001
   [junit4]   2> 42621 T99 oasc.CoreContainer.create Creating SolrCore 'core1' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/
   [junit4]   2> 42621 T99 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 42621 T98 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index/
   [junit4]   2> 42621 T99 oasc.SolrCore.<init> [core1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/
   [junit4]   2> 42621 T98 oasc.SolrCore.initIndex WARN [core0] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index' doesn't exist. Creating new index...
   [junit4]   2> 42621 T99 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 42622 T98 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index
   [junit4]   2> 42622 T99 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002
   [junit4]   2> 42623 T99 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index/
   [junit4]   2> 42623 T99 oasc.SolrCore.initIndex WARN [core1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index' doesn't exist. Creating new index...
   [junit4]   2> 42623 T99 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index
   [junit4]   2> 43436 T98 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 43436 T98 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 43436 T99 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index,segFN=segments_1,generation=1}
   [junit4]   2> 43437 T99 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 43438 T98 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 43439 T98 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 43439 T98 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 43439 T99 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 43439 T98 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 43439 T99 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 43440 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 43441 T99 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 43444 T98 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 43444 T98 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 43444 T99 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 43445 T99 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 43444 T98 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 43445 T99 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 43445 T98 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 43445 T99 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 43446 T98 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 43446 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 43446 T98 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 43446 T99 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 43448 T98 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 43448 T99 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 43451 T98 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 43451 T98 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 43451 T99 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 43451 T99 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 43455 T98 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 43455 T98 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 43455 T99 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index,segFN=segments_1,generation=1}
   [junit4]   2> 43456 T99 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 43456 T98 oass.SolrIndexSearcher.<init> Opening Searcher@1c753eb[core0] main
   [junit4]   2> 43456 T99 oass.SolrIndexSearcher.<init> Opening Searcher@1c2a6c3[core1] main
   [junit4]   2> 43456 T98 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/conf; switching to use InMemory storage instead.
   [junit4]   2> 43456 T98 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 43456 T99 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/conf; switching to use InMemory storage instead.
   [junit4]   2> 43457 T99 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 43457 T98 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 43457 T99 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 43457 T98 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 43458 T99 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 43458 T98 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 43458 T99 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 43458 T98 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 43458 T99 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 43459 T100 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@1c753eb[core0] main{UninvertingDirectoryReader()}
   [junit4]   2> 43459 T101 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@1c2a6c3[core1] main{UninvertingDirectoryReader()}
   [junit4]   2> 43459 T99 oasc.CoreContainer.registerCore registering core: core1
   [junit4]   2> 43459 T98 oasc.CoreContainer.registerCore registering core: core0
   [junit4]   2> 43460 T88 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1
   [junit4]   2> 43461 T88 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C37 name=core0 org.apache.solr.core.SolrCore@1ee2790
   [junit4]   2> 43632 T97 C37 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43632 T97 C37 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 43636 T97 C37 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 43636 T97 C37 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 43637 T97 C37 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {deleteByQuery=*:* (-1473207073329119232),commit=} 0 7
   [junit4]   2> ASYNC  NEW_CORE C38 name=core1 org.apache.solr.core.SolrCore@57ca3e
   [junit4]   2> 43754 T94 C38 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43754 T94 C38 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 43755 T94 C38 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 43755 T94 C38 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 43756 T94 C38 oasup.LogUpdateProcessor.finish [core1] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {deleteByQuery=*:* (-1473207073457045504),commit=} 0 4
   [junit4]   2> ASYNC  NEW_CORE C39 name=core0 org.apache.solr.core.SolrCore@1ee2790
   [junit4]   2> 43780 T96 C39 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43826 T96 C39 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_2,generation=2}
   [junit4]   2> 43826 T96 C39 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 43860 T96 C39 oass.SolrIndexSearcher.<init> Opening Searcher@172b62a[core0] main
   [junit4]   2> 43861 T100 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@172b62a[core0] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 43862 T96 C39 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 43862 T96 C39 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {add=[AAA (1473207073484308480)],commit=} 0 84
   [junit4]   2> 43882 T94 C38 oasup.LogUpdateProcessor.finish [core1] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {} 0 1
   [junit4]   2> 43883 T94 C38 oasc.SolrException.log Ignoring exception matching unknown field
   [junit4]   2> 43897 T97 C38 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43924 T97 C38 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index,segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index,segFN=segments_2,generation=2}
   [junit4]   2> 43925 T97 C38 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 43928 T97 C38 oass.SolrIndexSearcher.<init> Opening Searcher@39387f[core1] main
   [junit4]   2> 43929 T97 C38 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 43929 T101 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@39387f[core1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 43929 T97 C38 oasup.LogUpdateProcessor.finish [core1] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {add=[BBB (1473207073606991872)],commit=} 0 34
   [junit4]   2> 43941 T95 C39 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {} 0 0
   [junit4]   2> 43942 T95 C39 oasc.SolrException.log Ignoring exception matching unknown field
   [junit4]   2> 43955 T94 C39 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 43986 T94 C39 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_2,generation=2}
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_3,generation=3}
   [junit4]   2> 43986 T94 C39 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 43989 T94 C39 oass.SolrIndexSearcher.<init> Opening Searcher@36a561[core0] main
   [junit4]   2> 43990 T94 C39 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 43990 T100 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@36a561[core0] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1) Uninverting(_1(5.0):C1))}
   [junit4]   2> 43991 T94 C39 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {add=[BBB1 (1473207073668857856)],commit=} 0 37
   [junit4]   2> 44020 T96 C39 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q=id:AAA} hits=1 status=0 QTime=19 
   [junit4]   2> 44035 T94 C38 oasc.SolrCore.execute [core1] webapp=/example path=/select params={wt=javabin&version=2&q=id:AAA} hits=0 status=0 QTime=0 
   [junit4]   2> 44045 T96 C39 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q=id:AAA} hits=1 status=0 QTime=0 
   [junit4]   2> 44058 T94 C39 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q=id:BBB} hits=0 status=0 QTime=1 
   [junit4]   2> 44071 T96 C38 oasc.SolrCore.execute [core1] webapp=/example path=/select params={wt=javabin&version=2&q=id:AAA} hits=0 status=0 QTime=0 
   [junit4]   2> 44089 T94 C38 oasc.SolrCore.execute [core1] webapp=/example path=/select params={wt=javabin&version=2&q=id:BBB} hits=1 status=0 QTime=0 
   [junit4]   2> 44109 T96 C39 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q={!join+from%3Dtype+to%3Dname}*:*} hits=0 status=0 QTime=6 
   [junit4]   2> 44127 T94 C39 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q={!join+from%3Dtype+to%3Dname+fromIndex%3Dcore1}id:BBB} hits=2 status=0 QTime=3 
   [junit4]   2> ASYNC  NEW_CORE C40 name=core1 org.apache.solr.core.SolrCore@57ca3e
   [junit4]   2> 44155 T97 C40 oasc.SolrCore.execute [core1] webapp=/example path=/select params={wt=javabin&version=2&q={!join+from%3Dtype+to%3Dname+fromIndex%3Dcore0}id:AAA} hits=1 status=0 QTime=1 
   [junit4]   2> ASYNC  NEW_CORE C41 name=core0 org.apache.solr.core.SolrCore@1ee2790
   [junit4]   2> 44176 T95 C41 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q={!join+from%3Dtype+to%3Dname+fromIndex%3Dcore1}id:BB~} hits=2 status=0 QTime=10 
   [junit4]   2> 44215 T94 C41 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q={!join+from%3Dtype+to%3Dname+fromIndex%3Dcore1}core1:yup} hits=2 status=0 QTime=1 
   [junit4]   2> 44341 T95 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=STATUS&indexInfo=true&version=2&core=core0} status=0 QTime=8 
   [junit4]   2> 44364 T97 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/'
   [junit4]   2> 44415 T97 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 44482 T97 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 44483 T97 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 44491 T97 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 44506 T97 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 44507 T97 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 44508 T97 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 44510 T97 oasc.CoreContainer.reload Reloading SolrCore 'core0' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/
   [junit4]   2> 44510 T97 oasc.SolrCore.<init> [core0] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/
   [junit4]   2> 44511 T97 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 44511 T97 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index/
   [junit4]   2> 44513 T97 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 44513 T97 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 44514 T97 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 44514 T97 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 44514 T97 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 44515 T97 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 44515 T97 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 44515 T97 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 44516 T97 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 44516 T97 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 44521 T97 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 44525 T97 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 44525 T97 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 44526 T97 oass.SolrIndexSearcher.<init> Opening Searcher@b7cf19[core0] main
   [junit4]   2> 44527 T97 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/conf; switching to use InMemory storage instead.
   [junit4]   2> 44527 T97 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 44528 T97 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 44528 T97 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 44529 T97 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 44529 T97 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 44531 T97 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
   [junit4]   2> 44532 T103 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@b7cf19[core0] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1) Uninverting(_1(5.0):C1))}
   [junit4]   2> 44532 T97 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=core0
   [junit4]   2> 44533 T97 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=core0
   [junit4]   2> 44541 T97 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_3,generation=3}
   [junit4]   2> 44541 T97 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
   [junit4]   2> 44542 T97 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
   [junit4]   2> 44548 T97 oass.SolrIndexSearcher.<init> Opening Searcher@313223[core0] main
   [junit4]   2> 44549 T97 oasc.CoreContainer.registerCore replacing core: core0
   [junit4]   2> 44549 T103 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@313223[core0] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1) Uninverting(_1(5.0):C1))}
   [junit4]   2> 44549 T97 oasc.SolrCore.close [core0]  CLOSING SolrCore org.apache.solr.core.SolrCore@1ee2790
   [junit4]   2> 44550 T97 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=3,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=1,transaction_logs_total_size=330,transaction_logs_total_number=3}
   [junit4]   2> 44555 T97 oasc.SolrCore.closeSearcher [core0] Closing main searcher on request.
   [junit4]   2> 44555 T97 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RELOAD&indexInfo=true&version=2&core=core0} status=0 QTime=191 
   [junit4]   2> ASYNC  NEW_CORE C42 name=core0 org.apache.solr.core.SolrCore@e04922
   [junit4]   2> 44580 T97 C42 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q=id:AAA} hits=1 status=0 QTime=0 
   [junit4]   2> 44593 T95 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=STATUS&indexInfo=true&version=2&core=core0} status=0 QTime=1 
   [junit4]   2> 44603 T97 oasc.CoreContainer.registerCore registering core: corea
   [junit4]   2> 44604 T97 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RENAME&indexInfo=true&version=2&core=core1&other=corea} status=0 QTime=1 
   [junit4]   2> 44617 T96 oasc.CoreContainer.registerCore registering core: coreb
   [junit4]   2> 44618 T96 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RENAME&indexInfo=true&version=2&core=corea&other=coreb} status=0 QTime=1 
   [junit4]   2> 44629 T94 oasc.CoreContainer.registerCore registering core: corec
   [junit4]   2> 44630 T94 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RENAME&indexInfo=true&version=2&core=coreb&other=corec} status=0 QTime=1 
   [junit4]   2> 44642 T96 oasc.CoreContainer.registerCore registering core: cored
   [junit4]   2> 44643 T96 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RENAME&indexInfo=true&version=2&core=corec&other=cored} status=0 QTime=1 
   [junit4]   2> 44654 T95 oasc.CoreContainer.registerCore registering core: corefoo
   [junit4]   2> 44655 T95 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&action=RENAME&indexInfo=true&version=2&core=cored&other=corefoo} status=0 QTime=1 
   [junit4]   2> ASYNC  NEW_CORE C43 name=corefoo org.apache.solr.core.SolrCore@57ca3e
   [junit4]   2> 44854 T96 C43 oasc.SolrCore.execute [corefoo] webapp=/example path=/select params={wt=javabin&version=2&q=id:BBB} hits=1 status=0 QTime=0 
   [junit4]   2> 44945 T96 C43 oasc.SolrCore.execute [corefoo] webapp=/example path=/admin/system params={wt=javabin&version=2&qt=/admin/system} status=0 QTime=3 
   [junit4]   2> 45048 T96 C42 oasc.SolrCore.execute [core0] webapp=/example path=/admin/system params={wt=javabin&version=2&qt=/admin/system} status=0 QTime=5 
   [junit4]   2> 45072 T95 oasc.SolrCore.close [corefoo]  CLOSING SolrCore org.apache.solr.core.SolrCore@57ca3e
   [junit4]   2> 45073 T95 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=1,cumulative_adds=2,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=1,transaction_logs_total_size=198,transaction_logs_total_number=2}
   [junit4]   2> 45073 T95 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 45074 T95 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 45074 T95 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 45079 T95 oasc.SolrCore.closeSearcher [corefoo] Closing main searcher on request.
   [junit4]   2> 45080 T95 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 45080 T95 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002;done=false>>]
   [junit4]   2> 45081 T95 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002
   [junit4]   2> 45081 T95 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index;done=false>>]
   [junit4]   2> 45082 T95 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index
   [junit4]   2> 45082 T95 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-002/index
   [junit4]   2> 45097 T95 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&deleteInstanceDir=false&deleteDataDir=false&action=UNLOAD&indexInfo=true&version=2&deleteIndex=true&core=corefoo} status=0 QTime=25 
   [junit4]   2> 45109 T97 oasc.SolrCore.close [core0]  CLOSING SolrCore org.apache.solr.core.SolrCore@e04922
   [junit4]   2> 45110 T97 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=330,transaction_logs_total_number=3}
   [junit4]   2> 45110 T97 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 45110 T97 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 45111 T97 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 45115 T97 oasc.SolrCore.closeSearcher [core0] Closing main searcher on request.
   [junit4]   2> 45116 T97 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 45116 T97 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001;done=false>>]
   [junit4]   2> 45117 T97 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index;done=false>>, CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001;done=false>>]
   [junit4]   2> 45117 T97 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index
   [junit4]   2> 45117 T97 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001
   [junit4]   2> 45118 T97 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001 []
   [junit4]   2> 45118 T97 oasc.CachingDirectoryFactory.close Removing directory after core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001
   [junit4]   2> 45125 T97 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={wt=javabin&deleteInstanceDir=false&deleteDataDir=true&action=UNLOAD&indexInfo=true&version=2&deleteIndex=false&core=core0} status=0 QTime=16 
   [junit4]   2> 45127 T88 oas.SolrTestCaseJ4.tearDown ###Ending testMultiCore
   [junit4]   2> 45139 T88 oejuc.AbstractLifeCycle.setFailed WARN FAILED org.eclipse.jetty.server.nio.SelectChannelConnector$ConnectorSelectorManager@9f809d: java.util.ConcurrentModificationException java.util.ConcurrentModificationException
   [junit4]   2> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
   [junit4]   2> 	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
   [junit4]   2> 	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:884)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager.doStop(SelectorManager.java:330)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AggregateLifeCycle.doStop(AggregateLifeCycle.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:354)
   [junit4]   2> 	at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStop(SslSelectChannelConnector.java:642)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:338)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:457)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.tearDown(MultiCoreExampleJettyTest.java:63)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 45141 T88 oejuc.AbstractLifeCycle.setFailed WARN FAILED SslSelectChannelConnector@127.0.0.1:0: java.util.ConcurrentModificationException java.util.ConcurrentModificationException
   [junit4]   2> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
   [junit4]   2> 	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
   [junit4]   2> 	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:884)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager.doStop(SelectorManager.java:330)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AggregateLifeCycle.doStop(AggregateLifeCycle.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:354)
   [junit4]   2> 	at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStop(SslSelectChannelConnector.java:642)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:338)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:457)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.tearDown(MultiCoreExampleJettyTest.java:63)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 45158 T88 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=16940592
   [junit4]   2> 45159 T88 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/example,null}
   [junit4]   2> 45212 T88 oejuc.AbstractLifeCycle.setFailed WARN FAILED org.eclipse.jetty.server.Server@173c26d: java.util.ConcurrentModificationException java.util.ConcurrentModificationException
   [junit4]   2> 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
   [junit4]   2> 	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
   [junit4]   2> 	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:884)
   [junit4]   2> 	at org.eclipse.jetty.io.nio.SelectorManager.doStop(SelectorManager.java:330)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AggregateLifeCycle.doStop(AggregateLifeCycle.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:354)
   [junit4]   2> 	at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStop(SslSelectChannelConnector.java:642)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:338)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:457)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.tearDown(MultiCoreExampleJettyTest.java:63)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=MultiCoreExampleJettyTest -Dtests.method=testMultiCore -Dtests.seed=3C116687DE339825 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_AR -Dtests.timezone=America/Argentina/Mendoza -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   3.07s J1 | MultiCoreExampleJettyTest.testMultiCore <<<
   [junit4]    > Throwable #1: java.util.ConcurrentModificationException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3C116687DE339825:B8397D7411E09510]:0)
   [junit4]    > 	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:922)
   [junit4]    > 	at java.util.HashMap$KeyIterator.next(HashMap.java:956)
   [junit4]    > 	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)
   [junit4]    > 	at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.stop(SelectorManager.java:884)
   [junit4]    > 	at org.eclipse.jetty.io.nio.SelectorManager.doStop(SelectorManager.java:330)
   [junit4]    > 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]    > 	at org.eclipse.jetty.util.component.AggregateLifeCycle.doStop(AggregateLifeCycle.java:107)
   [junit4]    > 	at org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:354)
   [junit4]    > 	at org.eclipse.jetty.server.ssl.SslSelectChannelConnector.doStop(SslSelectChannelConnector.java:642)
   [junit4]    > 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]    > 	at org.eclipse.jetty.server.Server.doStop(Server.java:338)
   [junit4]    > 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.stop(JettySolrRunner.java:457)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.MultiCoreExampleJettyTest.tearDown(MultiCoreExampleJettyTest.java:63)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 45275 T88 oas.SolrTestCaseJ4.setUp ###Starting testDistributed
   [junit4]   2> 45276 T88 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 45281 T88 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 45284 T88 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:41854
   [junit4]   2> 45286 T88 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 45287 T88 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 45287 T88 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore
   [junit4]   2> 45288 T88 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/'
   [junit4]   2> 45301 T88 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/solr.xml
   [junit4]   2> 45335 T88 oasc.CoreContainer.<init> New CoreContainer 25990192
   [junit4]   2> 45335 T88 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/]
   [junit4]   2> 45336 T88 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 45337 T88 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 45337 T88 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 45337 T88 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 45337 T88 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 45338 T88 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 45338 T88 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 45338 T88 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 45339 T88 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 45341 T88 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 45342 T88 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 45342 T88 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 45346 T127 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/'
   [junit4]   2> 45346 T128 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/'
   [junit4]   2> 45393 T127 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 45397 T128 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_5_0
   [junit4]   2> 45437 T127 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 45437 T127 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 45442 T128 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 45443 T128 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 45444 T127 oass.IndexSchema.readSchema [core0] Schema name=example core zero
   [junit4]   2> 45449 T128 oass.IndexSchema.readSchema [core1] Schema name=example core one
   [junit4]   2> 45457 T127 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 45458 T127 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 45459 T127 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 45460 T127 oasc.CoreContainer.create Creating SolrCore 'core0' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/
   [junit4]   2> 45461 T127 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 45461 T127 oasc.SolrCore.<init> [core0] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/
   [junit4]   2> 45461 T127 oasc.SolrCore.<init> JMX monitoring not detected for core: core0
   [junit4]   2> 45462 T128 oass.IndexSchema.readSchema default search field in schema is name
   [junit4]   2> 45462 T127 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001
   [junit4]   2> 45462 T127 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index/
   [junit4]   2> 45462 T128 oass.IndexSchema.readSchema query parser default operator is OR
   [junit4]   2> 45463 T127 oasc.SolrCore.initIndex WARN [core0] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index' doesn't exist. Creating new index...
   [junit4]   2> 45463 T127 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index
   [junit4]   2> 45463 T128 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 45464 T128 oasc.CoreContainer.create Creating SolrCore 'core1' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/
   [junit4]   2> 45465 T128 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
   [junit4]   2> 45465 T128 oasc.SolrCore.<init> [core1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/
   [junit4]   2> 45466 T128 oasc.SolrCore.<init> JMX monitoring not detected for core: core1
   [junit4]   2> 45466 T128 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003
   [junit4]   2> 45466 T128 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index/
   [junit4]   2> 45467 T128 oasc.SolrCore.initIndex WARN [core1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index' doesn't exist. Creating new index...
   [junit4]   2> 45467 T128 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index
   [junit4]   2> 45567 T127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 45567 T127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45568 T128 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index,segFN=segments_1,generation=1}
   [junit4]   2> 45569 T128 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45569 T127 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 45569 T127 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 45570 T127 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 45570 T128 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 45570 T127 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 45570 T128 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 45571 T128 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 45571 T128 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 45570 T127 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 45571 T127 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 45571 T128 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 45572 T128 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.FieldAnalysisRequestHandler
   [junit4]   2> 45572 T127 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 45572 T128 oasc.RequestHandlers.initHandlersFromConfig created /analysis/field: solr.FieldAnalysisRequestHandler
   [junit4]   2> 45573 T127 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 45573 T128 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 45573 T127 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 45573 T128 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 45573 T127 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 45574 T128 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 45578 T127 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45578 T128 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 45580 T127 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 45580 T127 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 45580 T128 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 45580 T128 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 45582 T127 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 45582 T128 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index,segFN=segments_1,generation=1}
   [junit4]   2> 45582 T128 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45582 T127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 45583 T128 oass.SolrIndexSearcher.<init> Opening Searcher@9fa5a[core1] main
   [junit4]   2> 45583 T127 oass.SolrIndexSearcher.<init> Opening Searcher@4c3208[core0] main
   [junit4]   2> 45583 T128 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core1/conf; switching to use InMemory storage instead.
   [junit4]   2> 45583 T127 oasr.ManagedResourceStorage.newStorageIO WARN Cannot write to config directory /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/example/multicore/core0/conf; switching to use InMemory storage instead.
   [junit4]   2> 45583 T128 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 45584 T127 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 45584 T128 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 45584 T127 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 45585 T128 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 45585 T127 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 45585 T128 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 45585 T127 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
   [junit4]   2> 45586 T128 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 45586 T127 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 45587 T130 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@9fa5a[core1] main{UninvertingDirectoryReader()}
   [junit4]   2> 45587 T129 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@4c3208[core0] main{UninvertingDirectoryReader()}
   [junit4]   2> 45587 T127 oasc.CoreContainer.registerCore registering core: core0
   [junit4]   2> 45587 T128 oasc.CoreContainer.registerCore registering core: core1
   [junit4]   2> 45588 T88 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1
   [junit4]   2> 45588 T88 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C44 name=core0 org.apache.solr.core.SolrCore@738988
   [junit4]   2> 45690 T125 C44 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 45690 T125 C44 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 45691 T125 C44 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 45691 T125 C44 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 45692 T125 C44 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {deleteByQuery=*:* (-1473207075487088640),commit=} 0 4
   [junit4]   2> ASYNC  NEW_CORE C45 name=core1 org.apache.solr.core.SolrCore@640ce4
   [junit4]   2> 45794 T121 C45 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 45794 T121 C45 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 45795 T121 C45 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 45795 T121 C45 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 45796 T121 C45 oasup.LogUpdateProcessor.finish [core1] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {deleteByQuery=*:* (-1473207075596140544),commit=} 0 4
   [junit4]   2> 45807 T125 C44 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 45846 T125 C44 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index,segFN=segments_2,generation=2}
   [junit4]   2> 45846 T125 C44 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 45848 T125 C44 oass.SolrIndexSearcher.<init> Opening Searcher@1f6c651[core0] main
   [junit4]   2> 45849 T125 C44 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 45849 T129 oasc.SolrCore.registerSearcher [core0] Registered new searcher Searcher@1f6c651[core0] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 45849 T125 C44 oasup.LogUpdateProcessor.finish [core0] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {add=[core0 (1473207075610820608)],commit=} 0 43
   [junit4]   2> 45860 T121 C45 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 45930 T121 C45 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index,segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index,segFN=segments_2,generation=2}
   [junit4]   2> 45931 T121 C45 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 45933 T121 C45 oass.SolrIndexSearcher.<init> Opening Searcher@4749b6[core1] main
   [junit4]   2> 45934 T121 C45 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 45934 T130 oasc.SolrCore.registerSearcher [core1] Registered new searcher Searcher@4749b6[core1] main{UninvertingDirectoryReader(Uninverting(_0(5.0):C1))}
   [junit4]   2> 45935 T121 C45 oasup.LogUpdateProcessor.finish [core1] webapp=/example path=/update params={waitSearcher=true&wt=javabin&softCommit=false&version=2&commit=true} {add=[core1 (1473207075666395136)],commit=} 0 76
   [junit4]   2> 45946 T126 C44 oasc.SolrCore.execute [core0] webapp=/example path=/select params={wt=javabin&version=2&q=*:*} hits=1 status=0 QTime=0 
   [junit4]   2> 45956 T122 C45 oasc.SolrCore.execute [core1] webapp=/example path=/select params={wt=javabin&version=2&q=*:*} hits=1 status=0 QTime=1 
   [junit4]   2> 46066 T122 C44 oasc.SolrCore.execute [core0] webapp=/example path=/select params={NOW=1404959750922&wt=javabin&isShard=true&version=2&fsv=true&rows=10&shard.url=https://127.0.0.1:41854/example/core0&q=*:*&start=0&distrib=false&fl=id,score} hits=1 status=0 QTime=2 
   [junit4]   2> 46115 T125 C45 oasc.SolrCore.execute [core1] webapp=/example path=/select params={NOW=1404959750922&wt=javabin&isShard=true&version=2&fsv=true&rows=10&shard.url=https://127.0.0.1:41854/example/core1&q=*:*&start=0&distrib=false&fl=id,score} hits=1 status=0 QTime=1 
   [junit4]   2> ASYNC  NEW_CORE C46 name=core0 org.apache.solr.core.SolrCore@738988
   [junit4]   2> 46150 T126 C46 oasc.SolrCore.execute [core0] webapp=/example path=/select params={NOW=1404959750922&wt=javabin&isShard=true&version=2&ids=core0&shard.url=https://127.0.0.1:41854/example/core0&q=*:*&distrib=false&fl=id,s:[shard]} status=0 QTime=1 
   [junit4]   2> ASYNC  NEW_CORE C47 name=core1 org.apache.solr.core.SolrCore@640ce4
   [junit4]   2> 46156 T126 C47 oasc.SolrCore.execute [core1] webapp=/example path=/select params={NOW=1404959750922&wt=javabin&isShard=true&version=2&ids=core1&shard.url=https://127.0.0.1:41854/example/core1&q=*:*&distrib=false&fl=id,s:[shard]} status=0 QTime=0 
   [junit4]   2> 46158 T121 C46 oasc.SolrCore.execute [core0] webapp=/example path=/select params={shards=https://127.0.0.1:41854/example/core0,https://127.0.0.1:41854/example/core1&wt=javabin&version=2&q=*:*&fl=id,s:[shard]} hits=2 status=0 QTime=188 
   [junit4]   2> 46159 T88 oas.SolrTestCaseJ4.tearDown ###Ending testDistributed
   [junit4]   2> 46171 T88 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=25990192
   [junit4]   2> 46172 T88 oasc.SolrCore.close [core1]  CLOSING SolrCore org.apache.solr.core.SolrCore@640ce4
   [junit4]   2> 46172 T88 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=167,transaction_logs_total_number=2}
   [junit4]   2> 46173 T88 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 46173 T88 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 46174 T88 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 46176 T88 oasc.SolrCore.closeSearcher [core1] Closing main searcher on request.
   [junit4]   2> 46176 T88 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 46176 T88 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index;done=false>>]
   [junit4]   2> 46177 T88 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003/index
   [junit4]   2> 46177 T88 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003;done=false>>]
   [junit4]   2> 46177 T88 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-003
   [junit4]   2> 46178 T88 oasc.SolrCore.close [core0]  CLOSING SolrCore org.apache.solr.core.SolrCore@738988
   [junit4]   2> 46178 T88 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=167,transaction_logs_total_number=2}
   [junit4]   2> 46179 T88 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 46179 T88 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 46179 T88 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 46181 T88 oasc.SolrCore.closeSearcher [core0] Closing main searcher on request.
   [junit4]   2> 46182 T88 oasc.CachingDirectoryFactory.close Closing StandardDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 46182 T88 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index;done=false>>]
   [junit4]   2> 46182 T88 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001/index
   [junit4]   2> 46183 T88 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001;done=false>>]
   [junit4]   2> 46183 T88 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001/tempDir-001
   [junit4]   2> 46184 T88 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/example,null}
   [junit4]   2> 46237 T88 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-solrj/test/J1/./temp/solr.client.solrj.embedded.MultiCoreExampleJettyTest-3C116687DE339825-001
   [junit4]   2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=es_AR, timezone=America/Argentina/Mendoza
   [junit4]   2> NOTE: Linux 3.8.0-42-generic i386/Oracle Corporation 1.7.0_60 (32-bit)/cpus=8,threads=1,free=26997456,total=64880640
   [junit4]   2> NOTE: All tests run in this JVM: [TestSpellCheckResponse, SolrExampleBinaryTest, BasicHttpSolrServerTest, SolrDocumentTest, TestHash, SolrExampleStreamingTest, MultiCoreExampleJettyTest]
   [junit4] Completed on J1 in 4.07s, 2 tests, 1 error <<< FAILURES!

[...truncated 97 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1298: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:922: There were test failures: 49 suites, 295 tests, 1 error

Total time: 70 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_60 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any