You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/10/19 14:17:49 UTC
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 180 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/180/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.morphlines.solr.SolrMorphlineZkTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.morphlines.solr.SolrMorphlineZkTest: 1) Thread[id=117, name=OverseerHdfsCoreFailoverThread-96789205255913478-127.0.0.1:52212_solr-n_0000000001, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.morphlines.solr.SolrMorphlineZkTest:
1) Thread[id=117, name=OverseerHdfsCoreFailoverThread-96789205255913478-127.0.0.1:52212_solr-n_0000000001, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([70C3D2538F605853]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130) at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:74) at org.apache.solr.update.HdfsUpdateLog.ensureLog(HdfsUpdateLog.java:313) at org.apache.solr.update.UpdateLog.deleteByQuery(UpdateLog.java:490) at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:464) at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:78) at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:52) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:962) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionDeleteByQuery(DistributedUpdateProcessor.java:1457) at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1350) at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1159) at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:124) at org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:159) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:112) at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2171) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:74)
at org.apache.solr.update.HdfsUpdateLog.ensureLog(HdfsUpdateLog.java:313)
at org.apache.solr.update.UpdateLog.deleteByQuery(UpdateLog.java:490)
at org.apache.solr.update.DirectUpdateHandler2.deleteByQuery(DirectUpdateHandler2.java:464)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:78)
at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:52)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:962)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionDeleteByQuery(DistributedUpdateProcessor.java:1457)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doDeleteByQuery(DistributedUpdateProcessor.java:1350)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1159)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:124)
at org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:159)
at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:112)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2171)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([3DA7A1D64F873C7D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:261)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testPartialReplicationWithTruncatedTlog
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([3DA7A1D64F873C7D:107CEF8EE6D3C4FC]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.collectInfo(BaseCdcrDistributedZkTest.java:810)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.assertUpdateLogsEquals(CdcrReplicationHandlerTest.java:300)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testPartialReplicationWithTruncatedTlog(CdcrReplicationHandlerTest.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:35203
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:35203
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:437)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 47 more
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=193527, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=193527, name=collection0, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41241/w
at __randomizedtesting.SeedInfo.seed([3DA7A1D64F873C7D]:0)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41241/w
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:994)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 9 more
Build Log:
[...truncated 11422 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/init-core-data-001
[junit4] 2> 576781 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[3DA7A1D64F873C7D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 576782 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[3DA7A1D64F873C7D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 576783 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 576783 INFO (Thread-549) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 576784 INFO (Thread-549) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 576884 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkTestServer start zk server on port:33757
[junit4] 2> 576896 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 576898 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 576899 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 576900 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 576901 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 576901 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 576905 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 576906 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 576907 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 576908 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 576909 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 576913 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 576915 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 576916 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
[junit4] 2> 576917 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 576918 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 576919 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 576920 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 576921 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 576922 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 576922 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 576923 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 576924 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 576985 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/cores/collection1
[junit4] 2> 576986 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 576987 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c3f4632{/,null,AVAILABLE}
[junit4] 2> 576988 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.ServerConnector Started ServerConnector@71b3b959{HTTP/1.1,[http/1.1]}{127.0.0.1:53679}
[junit4] 2> 576988 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server Started @579390ms
[junit4] 2> 576989 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/tempDir-001/control/data, hostContext=/, hostPort=53679, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/cores}
[junit4] 2> 576989 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 576989 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 576989 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 576989 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 576989 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-19T12:54:29.074Z
[junit4] 2> 576992 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 576992 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/solr.xml
[junit4] 2> 576996 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 576997 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33757/solr
[junit4] 2> 577011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53679_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53679_
[junit4] 2> 577011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53679_ ] o.a.s.c.Overseer Overseer (id=96788917050998790-127.0.0.1:53679_-n_0000000000) starting
[junit4] 2> 577015 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53679_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53679_
[junit4] 2> 577016 INFO (OverseerStateUpdate-96788917050998790-127.0.0.1:53679_-n_0000000000) [n:127.0.0.1:53679_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 577028 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53679_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/cores
[junit4] 2> 577028 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53679_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 577031 INFO (OverseerStateUpdate-96788917050998790-127.0.0.1:53679_-n_0000000000) [n:127.0.0.1:53679_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 578039 WARN (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 578040 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 578051 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 578126 WARN (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 578128 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 578136 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 578136 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/control-001/cores/collection1/data/]
[junit4] 2> 578137 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b5d162b
[junit4] 2> 578137 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 578140 WARN (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 578145 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 578145 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 578146 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 578146 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 578146 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=94.0908203125, floorSegmentMB=2.0498046875, forceMergeDeletesPctAllowed=17.55091127559318, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 578146 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1342ca42[collection1] main]
[junit4] 2> 578147 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 578147 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 578147 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 578148 INFO (searcherExecutor-1111-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1342ca42[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 578148 INFO (coreLoadExecutor-1110-thread-1-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548622674246238208
[junit4] 2> 578154 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 578154 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 578154 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53679/collection1/
[junit4] 2> 578154 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 578154 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53679/collection1/ has no replicas
[junit4] 2> 578160 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53679/collection1/ shard1
[junit4] 2> 578311 INFO (coreZkRegister-1103-thread-1-processing-n:127.0.0.1:53679_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53679_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 578533 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 578534 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 578602 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/cores/collection1
[junit4] 2> 578602 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001
[junit4] 2> 578603 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 578604 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74deaadf{/,null,AVAILABLE}
[junit4] 2> 578604 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.ServerConnector Started ServerConnector@51b86160{HTTP/1.1,[http/1.1]}{127.0.0.1:35203}
[junit4] 2> 578604 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server Started @581006ms
[junit4] 2> 578605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=35203, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/cores}
[junit4] 2> 578605 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 578605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 578605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 578605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 578605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-19T12:54:30.690Z
[junit4] 2> 578608 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 578608 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/solr.xml
[junit4] 2> 578613 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 578613 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33757/solr
[junit4] 2> 578621 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:35203_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 578624 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:35203_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35203_
[junit4] 2> 578625 INFO (zkCallback-366-thread-3-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 578625 INFO (zkCallback-370-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 578626 INFO (zkCallback-375-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 578636 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:35203_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/cores
[junit4] 2> 578637 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:35203_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 578642 INFO (OverseerStateUpdate-96788917050998790-127.0.0.1:53679_-n_0000000000) [n:127.0.0.1:53679_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 579652 WARN (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 579652 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 579664 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 579736 WARN (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 579738 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 579747 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 579748 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 579748 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b5d162b
[junit4] 2> 579748 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 579751 WARN (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 579757 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 579757 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 579758 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 579758 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 579759 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=94.0908203125, floorSegmentMB=2.0498046875, forceMergeDeletesPctAllowed=17.55091127559318, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 579759 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b8c0028[collection1] main]
[junit4] 2> 579760 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 579760 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 579760 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 579761 INFO (searcherExecutor-1122-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7b8c0028[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 579761 INFO (coreLoadExecutor-1121-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548622675937591296
[junit4] 2> 579770 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 579770 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 579770 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35203/collection1/
[junit4] 2> 579770 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 579770 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35203/collection1/ has no replicas
[junit4] 2> 579772 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35203/collection1/ shard2
[junit4] 2> 579923 INFO (coreZkRegister-1116-thread-1-processing-n:127.0.0.1:35203_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:35203_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 580202 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/cores/collection1
[junit4] 2> 580202 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001
[junit4] 2> 580203 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 580204 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3857d549{/,null,AVAILABLE}
[junit4] 2> 580204 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.ServerConnector Started ServerConnector@39ca26d6{HTTP/1.1,[http/1.1]}{127.0.0.1:38993}
[junit4] 2> 580204 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server Started @582606ms
[junit4] 2> 580205 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=38993, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/cores}
[junit4] 2> 580205 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 580205 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 580205 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 580205 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 580205 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-19T12:54:32.290Z
[junit4] 2> 580208 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 580208 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/solr.xml
[junit4] 2> 580213 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 580213 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33757/solr
[junit4] 2> 580237 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:38993_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 580241 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:38993_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38993_
[junit4] 2> 580242 INFO (zkCallback-370-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 580243 INFO (zkCallback-375-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 580243 INFO (zkCallback-366-thread-2-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 580243 INFO (zkCallback-381-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 580253 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:38993_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/cores
[junit4] 2> 580253 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:38993_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 580255 INFO (OverseerStateUpdate-96788917050998790-127.0.0.1:53679_-n_0000000000) [n:127.0.0.1:53679_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 581268 WARN (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 581269 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 581283 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 581366 WARN (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 581368 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 581376 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 581377 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 581377 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b5d162b
[junit4] 2> 581378 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 581381 WARN (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 581389 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 581389 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 581391 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 581391 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 581391 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=94.0908203125, floorSegmentMB=2.0498046875, forceMergeDeletesPctAllowed=17.55091127559318, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 581392 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6640cb7b[collection1] main]
[junit4] 2> 581392 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 581393 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 581393 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 581393 INFO (searcherExecutor-1133-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6640cb7b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 581394 INFO (coreLoadExecutor-1132-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548622677649915904
[junit4] 2> 581400 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 581400 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 581400 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38993/collection1/
[junit4] 2> 581400 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 581400 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38993/collection1/ has no replicas
[junit4] 2> 581403 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38993/collection1/ shard1
[junit4] 2> 581553 INFO (coreZkRegister-1127-thread-1-processing-n:127.0.0.1:38993_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38993_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 581831 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/cores/collection1
[junit4] 2> 581831 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001
[junit4] 2> 581832 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 581834 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@aee16e8{/,null,AVAILABLE}
[junit4] 2> 581834 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.ServerConnector Started ServerConnector@30c0ccf5{HTTP/1.1,[http/1.1]}{127.0.0.1:53110}
[junit4] 2> 581834 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.e.j.s.Server Started @584236ms
[junit4] 2> 581835 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=53110, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/cores}
[junit4] 2> 581835 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581835 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 581835 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581835 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581835 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-19T12:54:33.920Z
[junit4] 2> 581846 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 581846 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/solr.xml
[junit4] 2> 581852 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 581852 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33757/solr
[junit4] 2> 581866 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53110_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 581868 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53110_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53110_
[junit4] 2> 581869 INFO (zkCallback-381-thread-1-processing-n:127.0.0.1:38993_) [n:127.0.0.1:38993_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581869 INFO (zkCallback-370-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581870 INFO (zkCallback-375-thread-1-processing-n:127.0.0.1:35203_) [n:127.0.0.1:35203_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581870 INFO (zkCallback-366-thread-2-processing-n:127.0.0.1:53679_) [n:127.0.0.1:53679_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581873 INFO (zkCallback-387-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 581882 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53110_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/cores
[junit4] 2> 581882 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[3DA7A1D64F873C7D]) [n:127.0.0.1:53110_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 581885 INFO (OverseerStateUpdate-96788917050998790-127.0.0.1:53679_-n_0000000000) [n:127.0.0.1:53679_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 582895 WARN (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 582896 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 582908 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 582981 WARN (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 582983 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 582996 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 582996 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_3DA7A1D64F873C7D-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 582996 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b5d162b
[junit4] 2> 582997 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 582999 WARN (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 583006 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 583006 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 583006 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 583006 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 583007 INFO (coreLoadExecutor-1143-thread-1-processing-n:127.0.0.1:53110_) [n:127.0.0.1:53110_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping clas
[...truncated too long message...]
Processor According to ZK I (id=96789205255913478-127.0.0.1:52212_solr-n_0000000001) am no longer a leader.
[junit4] 2> 11672 ERROR (OverseerCollectionConfigSetProcessor-96789205255913478-127.0.0.1:52212_solr-n_0000000001) [n:127.0.0.1:52212_solr ] o.a.s.c.OverseerTaskProcessor :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/collection-queue-work
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:339)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:336)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:336)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.fetchZkChildren(DistributedQueue.java:299)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskQueue.getTailId(OverseerTaskQueue.java:264)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:162)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 11672 INFO (OverseerCollectionConfigSetProcessor-96789205255913478-127.0.0.1:52212_solr-n_0000000001) [n:127.0.0.1:52212_solr ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=96789205255913478-127.0.0.1:52212_solr-n_0000000001) am no longer a leader.
[junit4] 2> 11683 INFO (jetty-launcher-1-thread-2-EventThread) [n:127.0.0.1:52212_solr ] o.a.z.ClientCnxn EventThread shut down
[junit4] 2> 11683 WARN (zkCallback-12-thread-2-processing-n:127.0.0.1:52212_solr) [n:127.0.0.1:52212_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 11710 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@309e3ba1{/solr,null,UNAVAILABLE}
[junit4] 2> 13135 WARN (zkCallback-11-thread-3-processing-n:127.0.0.1:45016_solr) [n:127.0.0.1:45016_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 13135 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ec8f3eb{/solr,null,UNAVAILABLE}
[junit4] 2> 13136 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.ZooKeeperServer shutting down
[junit4] 2> 13136 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.SessionTrackerImpl Shutting down
[junit4] 2> 13136 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.PrepRequestProcessor Shutting down
[junit4] 2> 13136 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.SyncRequestProcessor Shutting down
[junit4] 2> 13136 INFO (ProcessThread(sid:0 cport:-1):) [ ] o.a.z.s.PrepRequestProcessor PrepRequestProcessor exited loop!
[junit4] 2> 13136 INFO (SyncThread:0) [ ] o.a.z.s.SyncRequestProcessor SyncRequestProcessor exited!
[junit4] 2> 13136 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.FinalRequestProcessor shutdown of request processor complete
[junit4] 2> 13137 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:48408 48408
[junit4] 2> 13314 INFO (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxnFactory NIOServerCnxn factory exited run method
[junit4] 2> 13314 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.ZooKeeperServer shutting down
[junit4] 2> 13314 INFO (Thread-1) [ ] o.a.z.s.ZooKeeperServer shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.SessionTrackerImpl Shutting down
[junit4] 2> 13314 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.SessionTrackerImpl Shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.PrepRequestProcessor Shutting down
[junit4] 2> 13315 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.PrepRequestProcessor Shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.SyncRequestProcessor Shutting down
[junit4] 2> 13315 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.SyncRequestProcessor Shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.FinalRequestProcessor shutdown of request processor complete
[junit4] 2> 13315 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.z.s.FinalRequestProcessor shutdown of request processor complete
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:48408 48408
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.ZooKeeperServer shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.SessionTrackerImpl Shutting down
[junit4] 2> 13315 INFO (Thread-1) [ ] o.a.z.s.PrepRequestProcessor Shutting down
[junit4] 2> 13316 INFO (Thread-1) [ ] o.a.z.s.SyncRequestProcessor Shutting down
[junit4] 2> 13316 INFO (Thread-1) [ ] o.a.z.s.FinalRequestProcessor shutdown of request processor complete
[junit4] 2> 13317 WARN (Thread-1) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 4 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/overseer/queue
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/live_nodes
[junit4] 2> 4 /solr/collections
[junit4] 2> 3 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 13317 INFO (SUITE-SolrMorphlineZkTest-seed#[70C3D2538F605853]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> Oct 19, 2016 2:07:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> 13586 INFO (SessionTracker) [ ] o.a.z.s.SessionTrackerImpl SessionTrackerImpl exited loop!
[junit4] 2> Oct 19, 2016 2:08:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.morphlines.solr.SolrMorphlineZkTest:
[junit4] 2> 1) Thread[id=117, name=OverseerHdfsCoreFailoverThread-96789205255913478-127.0.0.1:52212_solr-n_0000000001, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Oct 19, 2016 2:08:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=117, name=OverseerHdfsCoreFailoverThread-96789205255913478-127.0.0.1:52212_solr-n_0000000001, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] 2> Oct 19, 2016 2:08:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62), sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=hr, timezone=Africa/Djibouti
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=250514656,total=327680000
[junit4] 2> NOTE: All tests run in this JVM: [SolrMorphlineZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SolrMorphlineZkTest -Dtests.seed=70C3D2538F605853 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=hr -Dtests.timezone=Africa/Djibouti -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | SolrMorphlineZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.morphlines.solr.SolrMorphlineZkTest:
[junit4] > 1) Thread[id=117, name=OverseerHdfsCoreFailoverThread-96789205255913478-127.0.0.1:52212_solr-n_0000000001, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([70C3D2538F605853]:0)
[junit4] Completed [4/4 (1!)] on J0 in 34.75s, 1 test, 1 error <<< FAILURES!
[...truncated 50344 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 200 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/200/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest
Error Message:
ObjectTracker found 5 object(s) that were not released!!! [TransactionLog, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, RawDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:188) at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:344) at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:859) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:428) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:415) at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:299) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:211) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:166) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:335) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:957) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1112) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:738) at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103) at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:298) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:263) at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:181) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107) at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2210) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:798) at org.apache.solr.core.SolrCore.reload(SolrCore.java:590) at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:944) at org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2716) at org.apache.solr.handler.SolrConfigHandler$Command.lambda$handleGET$0(SolrConfigHandler.java:223) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:332) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:641) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:847) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:673) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:847) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:841) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [TransactionLog, MDCAwareThreadPoolExecutor, MockDirectoryWrapper, MockDirectoryWrapper, RawDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:188)
at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:344)
at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:859)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:428)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:415)
at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:299)
at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:211)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:166)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:335)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:957)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1112)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:738)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:298)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:263)
at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:181)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186)
at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2210)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:798)
at org.apache.solr.core.SolrCore.reload(SolrCore.java:590)
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:944)
at org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2716)
at org.apache.solr.handler.SolrConfigHandler$Command.lambda$handleGET$0(SolrConfigHandler.java:223)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:332)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:641)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:847)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:66)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:673)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:847)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:260)
at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.SolrCloudExampleTest: 1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.SolrCloudExampleTest:
1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SolrCloudExampleTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:57395: Error CREATEing SolrCore 'test_unload_shard_and_collection_1': Unable to create core [test_unload_shard_and_collection_1] Caused by: Direct buffer memory
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57395: Error CREATEing SolrCore 'test_unload_shard_and_collection_1': Unable to create core [test_unload_shard_and_collection_1] Caused by: Direct buffer memory
at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105:BAAC37E0F907BCFD]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:125)
at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12114 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/init-core-data-001
[junit4] 2> 2692755 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 2692755 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 2692797 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 2692806 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2692812 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2692824 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_57390_hdfs____79s9xu/webapp
[junit4] 2> 2693225 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57390
[junit4] 2> 2693278 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2693279 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2693298 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58520_datanode____4vrgd3/webapp
[junit4] 2> 2693769 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58520
[junit4] 2> 2693860 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2693861 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2693874 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45362_datanode____.e5fzez/webapp
[junit4] 2> 2693952 INFO (IPC Server handler 3 on 49852) [ ] BlockStateChange BLOCK* processReport: from storage DS-676f556b-a2a1-4570-9b2a-a4e996ba6fa3 node DatanodeRegistration(127.0.0.1:50668, datanodeUuid=e6e53022-14f7-4b81-bb60-0c083e240a6a, infoPort=54399, infoSecurePort=0, ipcPort=52145, storageInfo=lv=-56;cid=testClusterID;nsid=1434397911;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2693953 INFO (IPC Server handler 3 on 49852) [ ] BlockStateChange BLOCK* processReport: from storage DS-712aff59-0a4b-44ba-ad1a-46eff0c2ddcb node DatanodeRegistration(127.0.0.1:50668, datanodeUuid=e6e53022-14f7-4b81-bb60-0c083e240a6a, infoPort=54399, infoSecurePort=0, ipcPort=52145, storageInfo=lv=-56;cid=testClusterID;nsid=1434397911;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
[junit4] 2> 2694329 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[32F8083A57FBD105]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45362
[junit4] 2> 2694449 INFO (IPC Server handler 4 on 49852) [ ] BlockStateChange BLOCK* processReport: from storage DS-4247a765-ab99-4430-942f-0bdc3039100d node DatanodeRegistration(127.0.0.1:59175, datanodeUuid=248f3a02-7435-48d0-854d-35b0b9db5e52, infoPort=39552, infoSecurePort=0, ipcPort=41757, storageInfo=lv=-56;cid=testClusterID;nsid=1434397911;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2694449 INFO (IPC Server handler 4 on 49852) [ ] BlockStateChange BLOCK* processReport: from storage DS-5c5cc759-d5bd-4af2-9102-a716f7766918 node DatanodeRegistration(127.0.0.1:59175, datanodeUuid=248f3a02-7435-48d0-854d-35b0b9db5e52, infoPort=39552, infoSecurePort=0, ipcPort=41757, storageInfo=lv=-56;cid=testClusterID;nsid=1434397911;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 2694495 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2694495 INFO (Thread-6358) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2694495 INFO (Thread-6358) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2694595 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.ZkTestServer start zk server on port:39566
[junit4] 2> 2694607 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2694609 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2694612 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2694613 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2694615 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2694616 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2694618 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2694619 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2694621 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2694622 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2694624 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2694736 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/control-001/cores/collection1
[junit4] 2> 2694737 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2694743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59a81e1e{/,null,AVAILABLE}
[junit4] 2> 2694743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.ServerConnector Started ServerConnector@6f5c145f{HTTP/1.1,[http/1.1]}{127.0.0.1:40556}
[junit4] 2> 2694743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.Server Started @2699365ms
[junit4] 2> 2694743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:49852/hdfs__localhost_49852__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001_tempDir-002_control_data, hostContext=/, hostPort=40556, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/control-001/cores}
[junit4] 2> 2694743 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2694744 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 2694744 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2694744 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2694744 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-16T04:22:38.699Z
[junit4] 2> 2694754 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1586b5f50730002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2694755 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2694755 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/control-001/solr.xml
[junit4] 2> 2694761 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2694762 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39566/solr
[junit4] 2> 2694764 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1586b5f50730003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2694792 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:40556_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40556_
[junit4] 2> 2694793 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:40556_ ] o.a.s.c.Overseer Overseer (id=96945449104244740-127.0.0.1:40556_-n_0000000000) starting
[junit4] 2> 2694809 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:40556_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40556_
[junit4] 2> 2694811 INFO (zkCallback-2642-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2694825 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:40556_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/control-001/cores
[junit4] 2> 2694825 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:40556_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2694839 INFO (OverseerStateUpdate-96945449104244740-127.0.0.1:40556_-n_0000000000) [n:127.0.0.1:40556_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2694861 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2694876 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2695545 WARN (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2695550 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2695570 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2695571 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49852/solr_hdfs_home
[junit4] 2> 2695571 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2695571 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/control-001/cores/collection1], dataDir=[hdfs://localhost:49852/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 2695571 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@459aa36
[junit4] 2> 2695571 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 2695584 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 2695584 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 2695584 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 2695598 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 2695599 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 2695617 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 2695624 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 2695624 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 2695624 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 2695631 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 2695632 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=279456669, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2695654 INFO (IPC Server handler 7 on 49852) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59175 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-712aff59-0a4b-44ba-ad1a-46eff0c2ddcb:NORMAL:127.0.0.1:50668|RBW], ReplicaUC[[DISK]DS-5c5cc759-d5bd-4af2-9102-a716f7766918:NORMAL:127.0.0.1:59175|FINALIZED]]} size 0
[junit4] 2> 2695655 INFO (IPC Server handler 7 on 49852) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50668 is added to blk_1073741825_1001 size 71
[junit4] 2> 2695664 WARN (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2695683 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 2695683 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2695683 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 2695703 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2695703 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2695710 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5888295815943397]
[junit4] 2> 2695722 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e6dfa16[collection1] main]
[junit4] 2> 2695723 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2695724 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2695724 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2695725 INFO (searcherExecutor-6251-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3e6dfa16[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2695725 INFO (coreLoadExecutor-6250-thread-1-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1551127186956615680
[junit4] 2> 2695733 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2695733 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2695733 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40556/collection1/
[junit4] 2> 2695733 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2695733 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40556/collection1/ has no replicas
[junit4] 2> 2695737 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40556/collection1/ shard1
[junit4] 2> 2695888 INFO (coreZkRegister-6243-thread-1-processing-n:127.0.0.1:40556_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40556_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2695938 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2695940 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2695940 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 2696055 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001/cores/collection1
[junit4] 2> 2696056 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001
[junit4] 2> 2696057 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2696058 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e51c2c6{/,null,AVAILABLE}
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4dcb18bc{HTTP/1.1,[http/1.1]}{127.0.0.1:57395}
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.e.j.s.Server Started @2700680ms
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:49852/hdfs__localhost_49852__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=57395, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001/cores}
[junit4] 2> 2696059 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2696059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2696060 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-16T04:22:40.015Z
[junit4] 2> 2696070 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2696070 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001/solr.xml
[junit4] 2> 2696076 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2696077 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39566/solr
[junit4] 2> 2696087 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:57395_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2696091 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:57395_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57395_
[junit4] 2> 2696093 INFO (zkCallback-2642-thread-3-processing-n:127.0.0.1:40556_) [n:127.0.0.1:40556_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2696093 INFO (zkCallback-2646-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2696098 INFO (zkCallback-2652-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2696116 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:57395_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001/cores
[junit4] 2> 2696116 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[32F8083A57FBD105]) [n:127.0.0.1:57395_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2696118 INFO (OverseerStateUpdate-96945449104244740-127.0.0.1:40556_-n_0000000000) [n:127.0.0.1:40556_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 2696221 INFO (zkCallback-2652-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2697130 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2697163 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2697261 WARN (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2697264 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2697288 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2697288 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49852/solr_hdfs_home
[junit4] 2> 2697288 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2697289 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_32F8083A57FBD105-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:49852/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 2697289 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@459aa36
[junit4] 2> 2697289 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 2697297 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 2697297 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 2697297 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 2697303 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 2697304 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 2697321 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49852/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 2697328 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 2697328 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 2697328 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 2697334 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 2697334 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=279456669, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2697348 INFO (IPC Server handler 9 on 49852) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50668 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4247a765-ab99-4430-942f-0bdc3039100d:NORMAL:127.0.0.1:59175|RBW], ReplicaUC[[DISK]DS-676f556b-a2a1-4570-9b2a-a4e996ba6fa3:NORMAL:127.0.0.1:50668|RBW]]} size 0
[junit4] 2> 2697350 INFO (IPC Server handler 8 on 49852) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59175 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4247a765-ab99-4430-942f-0bdc3039100d:NORMAL:127.0.0.1:59175|RBW], ReplicaUC[[DISK]DS-676f556b-a2a1-4570-9b2a-a4e996ba6fa3:NORMAL:127.0.0.1:50668|RBW]]} size 0
[junit4] 2> 2697362 WARN (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2697381 INFO (coreLoadExecutor-6261-thread-1-processing-n:127.0.0.1:57395_) [n:127.0.0.1:57395_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit
[...truncated too long message...]
156)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:664)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:260)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.SolrCloudExampleTest:
[junit4] > 1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=28333, name=searcherExecutor-10915-thread-1, state=WAITING, group=TGRP-SolrCloudExampleTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([32F8083A57FBD105]:0)
[junit4] Completed [491/653 (2!)] on J2 in 215.88s, 1 test, 1 failure, 2 errors <<< FAILURES!
[...truncated 61144 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 199 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/199/
13 tests failed.
FAILED: org.apache.lucene.search.TestFuzzyQuery.testRandom
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([DBDEA4DBCB14C93B]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([DBDEA4DBCB14C93B]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testBatchBoundaries
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0:B42819AB46DCA317]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:795)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testBatchBoundaries(CdcrReplicationDistributedZkTest.java:557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.TestCloudPivotFacet.test
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
Error Message:
Captured an uncaught exception in thread: Thread[id=25228, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25228, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
Error Message:
Captured an uncaught exception in thread: Thread[id=25293, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25293, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
Error Message:
Captured an uncaught exception in thread: Thread[id=25442, name=zkCallback-3531-thread-6, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25442, name=zkCallback-3531-thread-6, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet
Error Message:
Captured an uncaught exception in thread: Thread[id=25456, name=zkCallback-3531-thread-11, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25456, name=zkCallback-3531-thread-11, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_idx
Error Message:
There are still nodes recoverying - waited for 330 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0:37BB536E3A4CF11]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:184)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.waitForRecoveriesToFinish(TestStressCloudBlindAtomicUpdates.java:459)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:304)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_idx(TestStressCloudBlindAtomicUpdates.java:224)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler
Error Message:
ObjectTracker found 11 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 11 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:260)
at sun.reflect.GeneratedMethodAccessor42.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={ "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-b332ffff", "state":"active", "replicas":{}}, "shard2":{ "range":"b3330000-e665ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"e6660000-1998ffff", "state":"active", "replicas":{}}, "shard4":{ "range":"19990000-4ccbffff", "state":"active", "replicas":{}}, "shard5":{ "range":"4ccc0000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"4", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={
"replicationFactor":"3",
"shards":{
"shard1":{
"range":"80000000-b332ffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"b3330000-e665ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"e6660000-1998ffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"19990000-4ccbffff",
"state":"active",
"replicas":{}},
"shard5":{
"range":"4ccc0000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"4",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0:DE72B44A394B4D65]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0:BB798FA4C85CD8FC]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 1716 lines...]
[junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> nov 14, 2016 8:09:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=4874, name=TEST-TestFuzzyQuery.testRandom-seed#[DBDEA4DBCB14C93B], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4) Thread[id=4873, name=SUITE-TestFuzzyQuery-seed#[DBDEA4DBCB14C93B], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.method=testRandom -Dtests.seed=DBDEA4DBCB14C93B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=et-EE -Dtests.timezone=Europe/Belgrade -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7200s J1 | TestFuzzyQuery.testRandom <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([DBDEA4DBCB14C93B]:0)
[junit4] 2> nov 14, 2016 8:09:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> nov 14, 2016 8:09:56 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestFuzzyQuery:
[junit4] 2> 1) Thread[id=4874, name=TEST-TestFuzzyQuery.testRandom-seed#[DBDEA4DBCB14C93B], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> nov 14, 2016 8:09:56 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=4874, name=TEST-TestFuzzyQuery.testRandom-seed#[DBDEA4DBCB14C93B], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> nov 14, 2016 8:09:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=4874, name=TEST-TestFuzzyQuery.testRandom-seed#[DBDEA4DBCB14C93B], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Object.hashCode(Native Method)
[junit4] 2> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:426)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false,coord=no): {field=DFR GL2}, locale=et-EE, timezone=Europe/Belgrade
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=117523520,total=265289728
[junit4] 2> NOTE: All tests run in this JVM: [Test2BSortedDocValuesFixedSorted, TestBasics, TestAllFilesDetectTruncation, TestPrefixInBooleanQuery, TestSimilarity2, TestIndexWriterExceptions, TestTragicIndexWriterDeadlock, TestNativeFSLockFactory, TestBooleanQuery, TestSpanBoostQuery, TestIndexingSequenceNumbers, TestParallelReaderEmptyIndex, TestSubScorerFreqs, TestPerFieldDocValuesFormat, TestHighCompressionMode, TestSegmentReader, TestPostingsOffsets, TestPerFieldPostingsFormat, TestTermQuery, TestIndexInput, TestPagedBytes, TestFSTs, TestCachingCollector, TestOfflineSorter, TestSimpleFSLockFactory, TestLucene50CompoundFormat, TestBKD, TestTopDocsMerge, TestDateSort, TestIndexWriterOnJRECrash, TestSpanNearQuery, TestSortedSetSelector, TestSpanTermQuery, TestNoMergeScheduler, TestField, TestNRTReaderCleanup, TestIndexReaderClose, TestFuzzyQuery]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.seed=DBDEA4DBCB14C93B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=et-EE -Dtests.timezone=Europe/Belgrade -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestFuzzyQuery (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([DBDEA4DBCB14C93B]:0)
[junit4] Completed [406/441 (1!)] on J1 in 7223.17s, 9 tests, 2 errors <<< FAILURES!
[...truncated 10171 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/init-core-data-001
[junit4] 2> 354805 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[9607C0FE3F7862F0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 354805 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[9607C0FE3F7862F0]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /vt_n/l
[junit4] 2> 354809 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 354810 INFO (Thread-2177) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 354810 INFO (Thread-2177) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 354910 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.ZkTestServer start zk server on port:54451
[junit4] 2> 354926 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 354928 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 354930 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 354932 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 354933 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 354935 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 354936 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 354938 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 354939 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 354941 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 354951 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 354953 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteCreateSourceCollection
[junit4] 2> 355057 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/cores/collection1
[junit4] 2> 355058 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 355060 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1608a4e8{/vt_n/l,null,AVAILABLE}
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.ServerConnector Started ServerConnector@71759bb1{HTTP/1.1,[http/1.1]}{127.0.0.1:43222}
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.Server Started @358924ms
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/vt_n/l, hostPort=43222, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/cores}
[junit4] 2> 355063 ERROR (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 355063 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-14T21:38:51.349Z
[junit4] 2> 355069 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 355069 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/solr.xml
[junit4] 2> 355075 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 355076 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54451/solr
[junit4] 2> 355106 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43222_vt_n%2Fl
[junit4] 2> 355108 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.Overseer Overseer (id=96938199029972996-127.0.0.1:43222_vt_n%2Fl-n_0000000000) starting
[junit4] 2> 355116 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43222_vt_n%2Fl
[junit4] 2> 355123 INFO (zkCallback-472-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 355128 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/cores
[junit4] 2> 355128 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 355130 INFO (OverseerStateUpdate-96938199029972996-127.0.0.1:43222_vt_n%2Fl-n_0000000000) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 356136 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.ZkController Only one config set found in zk - using it:conf1
[junit4] 2> 356149 WARN (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 356150 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 356162 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 356345 WARN (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 356347 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 356381 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 356382 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-001/cores/collection1/data/]
[junit4] 2> 356382 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a69d9d8
[junit4] 2> 356384 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=11, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=13.2294921875, floorSegmentMB=1.416015625, forceMergeDeletesPctAllowed=18.607167246940108, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8086589340136101
[junit4] 2> 356478 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 356478 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 356479 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 356479 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 356479 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=0.3349609375, floorSegmentMB=1.0556640625, forceMergeDeletesPctAllowed=17.178059612439547, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 356480 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@586981d7[collection1] main]
[junit4] 2> 356483 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 356484 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 356488 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 356495 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 356501 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
[junit4] 2> 356552 INFO (searcherExecutor-1149-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@586981d7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 356552 INFO (coreLoadExecutor-1148-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1551011186305138688
[junit4] 2> 356602 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 356602 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 356602 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43222/vt_n/l/collection1/
[junit4] 2> 356602 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 356602 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43222/vt_n/l/collection1/ has no replicas
[junit4] 2> 356606 INFO (zkCallback-472-thread-2-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard2
[junit4] 2> 356610 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43222/vt_n/l/collection1/ shard2
[junit4] 2> 356739 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-002/cores/collection1
[junit4] 2> 356741 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 356742 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@294a168b{/vt_n/l,null,AVAILABLE}
[junit4] 2> 356743 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.ServerConnector Started ServerConnector@39fb6591{HTTP/1.1,[http/1.1]}{127.0.0.1:56465}
[junit4] 2> 356743 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.e.j.s.Server Started @360605ms
[junit4] 2> 356743 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/vt_n/l, hostPort=56465, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-002/cores}
[junit4] 2> 356743 ERROR (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 356744 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 356744 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 356744 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 356744 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-14T21:38:53.030Z
[junit4] 2> 356763 INFO (coreZkRegister-1141-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:43222_vt_n%2Fl c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 356774 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 356774 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-002/solr.xml
[junit4] 2> 356783 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 356783 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54451/solr
[junit4] 2> 356800 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:56465_vt_n%2Fl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 356805 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:56465_vt_n%2Fl ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56465_vt_n%2Fl
[junit4] 2> 356806 INFO (zkCallback-472-thread-1-processing-n:127.0.0.1:43222_vt_n%2Fl) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 356812 INFO (zkCallback-478-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 356823 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:56465_vt_n%2Fl ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_9607C0FE3F7862F0-001/jetty-002/cores
[junit4] 2> 356823 INFO (TEST-CdcrReplicationDistributedZkTest.testDeleteCreateSourceCollection-seed#[9607C0FE3F7862F0]) [n:127.0.0.1:56465_vt_n%2Fl ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 356829 INFO (OverseerStateUpdate-96938199029972996-127.0.0.1:43222_vt_n%2Fl-n_0000000000) [n:127.0.0.1:43222_vt_n%2Fl ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 357851 WARN (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 357852 INFO (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 357890 INFO (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 358026 WARN (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 358041 INFO (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 358062 INFO (coreLoadExecutor-1160-thread-1-processing-n:127.0.0.1:56465_vt_n%2Fl) [n:127.0.0.1:56465_vt_n%2Fl c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 358062 INFO (coreLoa
[...truncated too long message...]
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {pivot_dt1=Lucene50(blocksize=128), dense_pivot_i1=PostingsFormat(name=Direct), pivot_l1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_l_dv=PostingsFormat(name=Memory doPackFST= true), multiDefault=Lucene50(blocksize=128), pivot_f1=Lucene50(blocksize=128), intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_d1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_x_s=PostingsFormat(name=Memory doPackFST= true), pivot_b1=Lucene50(blocksize=128), pivot_dt=PostingsFormat(name=Direct), dense_pivot_ti1=PostingsFormat(name=Memory doPackFST= true), range_facet_l=Lucene50(blocksize=128), pivot_x_s1=Lucene50(blocksize=128), pivot_y_s=Lucene50(blocksize=128), pivot_z_s=PostingsFormat(name=Direct), dense_pivot_ti=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_z_s1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_tl1=Lucene50(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true), dense_pivot_x_s1=Lucene50(blocksize=128), timestamp=Lucene50(blocksize=128), dense_pivot_y_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_td1=Lucene50(blocksize=128), pivot_tdt=PostingsFormat(name=Memory doPackFST= true), pivot_i1=PostingsFormat(name=Direct), pivot_tf1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_td=PostingsFormat(name=Direct), pivot_tf=PostingsFormat(name=Memory doPackFST= true), pivot_f=PostingsFormat(name=Direct), pivot_ti=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), pivot_i=Lucene50(blocksize=128), pivot_tl=PostingsFormat(name=Direct), pivot_l=PostingsFormat(name=Memory doPackFST= true), pivot_y_s1=PostingsFormat(name=Direct), pivot_ti1=PostingsFormat(name=Memory doPackFST= true), range_facet_i_dv=Lucene50(blocksize=128), dense_pivot_i=Lucene50(blocksize=128), pivot_b=PostingsFormat(name=Direct), dense_pivot_y_s1=PostingsFormat(name=Direct), pivot_d=PostingsFormat(name=Memory doPackFST= true), pivot_tdt1=Lucene50(blocksize=128), dense_pivot_x_s=PostingsFormat(name=Memory doPackFST= true)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=381, maxMBSortInHeap=6.521166944707231, sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=mt-MT, timezone=SystemV/PST8PDT
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=286011528,total=477626368
[junit4] 2> NOTE: All tests run in this JVM: [DeleteStatusTest, TestSchemaSimilarityResource, URLClassifyProcessorTest, ClassificationUpdateProcessorFactoryTest, TestHdfsCloudBackupRestore, TestCloudSchemaless, TestCustomStream, DistributedFacetExistsSmallTest, LoggingHandlerTest, CollectionsAPIAsyncDistributedZkTest, CollectionReloadTest, StatsReloadRaceTest, TestCustomDocTransformer, TestRTimerTree, TestSmileRequest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SaslZkACLProviderTest, SpellCheckCollatorTest, SimplePostToolTest, TestSizeLimitedDistributedMap, DistributedMLTComponentTest, IndexBasedSpellCheckerTest, CdcrUpdateLogTest, VMParamsZkACLAndCredentialsProvidersTest, ExitableDirectoryReaderTest, RankQueryTest, DistributedFacetPivotSmallAdvancedTest, TestOnReconnectListenerSupport, BlockJoinFacetRandomTest, CoreAdminCreateDiscoverTest, SpellCheckComponentTest, TestInfoStreamLogging, ZkCLITest, DateMathParserTest, FileUtilsTest, TestConfigReload, TestSolrDynamicMBean, SpatialFilterTest, TestSearchPerf, DistributedIntervalFacetingTest, TestFuzzyAnalyzedSuggestions, RemoteQueryErrorTest, TestDocBasedVersionConstraints, TestAnalyzeInfixSuggestions, TestUninvertingReader, HighlighterTest, TestElisionMultitermQuery, OpenCloseCoreStressTest, JsonLoaderTest, SolrIndexConfigTest, TestBulkSchemaConcurrent, TestQuerySenderListener, BaseCdcrDistributedZkTest, TestIntervalFaceting, TestJsonFacets, UtilsToolTest, SchemaVersionSpecificBehaviorTest, CopyFieldTest, TestFiltering, SortByFunctionTest, TestDistributedStatsComponentCardinality, TestAuthorizationFramework, TestSlowCompositeReaderWrapper, CdcrReplicationHandlerTest, TestFreeTextSuggestions, DirectSolrConnectionTest, DistributedQueryComponentCustomSortTest, TestSolrIndexConfig, CollectionsAPIDistributedZkTest, IndexSchemaRuntimeFieldTest, CSVRequestHandlerTest, MultiTermTest, OutOfBoxZkACLAndCredentialsProvidersTest, TermVectorComponentTest, OverseerTaskQueueTest, TestRequestStatusCollectionAPI, TestCoreContainer, ZkSolrClientTest, TestGraphMLResponseWriter, SampleTest, TestOverriddenPrefixQueryForCustomFieldType, CachingDirectoryFactoryTest, TestFieldCollectionResource, TestMaxScoreQueryParser, TestCustomSort, TestDistributedGrouping, TestReqParamsAPI, SoftAutoCommitTest, TestCursorMarkWithoutUniqueKey, TestUseDocValuesAsStored, AliasIntegrationTest, EchoParamsTest, TestDFISimilarityFactory, JSONWriterTest, TestCloudPseudoReturnFields, MoreLikeThisHandlerTest, FacetPivotSmallTest, BitVectorTest, TestRebalanceLeaders, TestStressVersions, ZkStateWriterTest, TestManagedResource, TestFieldCacheReopen, DocumentBuilderTest, InfoHandlerTest, ChaosMonkeySafeLeaderTest, ZkControllerTest, HLLSerializationTest, BasicDistributedZkTest, SyncSliceTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, CdcrVersionReplicationTest, ConfigSetsAPITest, ConnectionManagerTest, CreateCollectionCleanupTest, CustomCollectionTest, DistributedQueueTest, ForceLeaderTest, PeerSyncReplicationTest, SSLMigrationTest, SharedFSAutoReplicaFailoverTest, TestCloudInspectUtil, TestCloudPivotFacet]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudPivotFacet -Dtests.seed=9607C0FE3F7862F0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=mt-MT -Dtests.timezone=SystemV/PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestCloudPivotFacet (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([9607C0FE3F7862F0]:0)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25228, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceededThrowable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25293, name=solr-idle-connections-evictor, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceededThrowable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25442, name=zkCallback-3531-thread-6, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceededThrowable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=25456, name=zkCallback-3531-thread-11, state=RUNNABLE, group=TGRP-TestCloudPivotFacet]
[junit4] > Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Completed [651/651 (5!)] on J1 in 7223.53s, 1 test, 6 errors <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161114_213252_370.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/heapdumps/java_pid19746.hprof ...
[junit4] Heap dump file created [661415171 bytes in 5.225 secs]
[junit4] <<< JVM J1: EOF ----
[junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161114_213252_370.syserr
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4]
[junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "solr-idle-connections-evictor"
[junit4] <<< JVM J1: EOF ----
[...truncated 11055 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:717: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid19746.hprof
Total time: 483 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 198 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/198/
4 tests failed.
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget
Error Message:
Timeout while trying to assert number of documents @ target_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ target_collection
at __randomizedtesting.SeedInfo.seed([B6A88B4914AF140A:16CA19D3C4F6D14F]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testResilienceWithDeleteByQueryOnTarget(CdcrReplicationDistributedZkTest.java:595)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: expected:<50> but was:<0>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1479074097480,generation=2,filelist=[_ml.cfe, _ml.cfs, _ml.si, _mm.cfe, _mm.cfs, _mm.si, _mo.fdt, _mo.fdx, _mo.fnm, _mo.nvd, _mo.nvm, _mo.si, _mo_FSTOrd50_0.doc, _mo_FSTOrd50_0.tbk, _mo_FSTOrd50_0.tix, _mp.cfe, _mp.cfs, _mp.si, _mr.cfe, _mr.cfs, _mr.si, segments_2]}]> but was:<[{indexVersion=1479074097480,generation=2,filelist=[_ml.cfe, _ml.cfs, _ml.si, _mm.cfe, _mm.cfs, _mm.si, _mo.fdt, _mo.fdx, _mo.fnm, _mo.nvd, _mo.nvm, _mo.si, _mo_FSTOrd50_0.doc, _mo_FSTOrd50_0.tbk, _mo_FSTOrd50_0.tix, _mp.cfe, _mp.cfs, _mp.si, _mr.cfe, _mr.cfs, _mr.si, segments_2]}, {indexVersion=1479074097480,generation=3,filelist=[_mp.cfe, _mp.cfs, _mp.si, _mq.cfe, _mq.cfs, _mq.si, _mr.cfe, _mr.cfs, _mr.si, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1479074097480,generation=2,filelist=[_ml.cfe, _ml.cfs, _ml.si, _mm.cfe, _mm.cfs, _mm.si, _mo.fdt, _mo.fdx, _mo.fnm, _mo.nvd, _mo.nvm, _mo.si, _mo_FSTOrd50_0.doc, _mo_FSTOrd50_0.tbk, _mo_FSTOrd50_0.tix, _mp.cfe, _mp.cfs, _mp.si, _mr.cfe, _mr.cfs, _mr.si, segments_2]}]> but was:<[{indexVersion=1479074097480,generation=2,filelist=[_ml.cfe, _ml.cfs, _ml.si, _mm.cfe, _mm.cfs, _mm.si, _mo.fdt, _mo.fdx, _mo.fnm, _mo.nvd, _mo.nvm, _mo.si, _mo_FSTOrd50_0.doc, _mo_FSTOrd50_0.tbk, _mo_FSTOrd50_0.tix, _mp.cfe, _mp.cfs, _mp.si, _mr.cfe, _mr.cfs, _mr.si, segments_2]}, {indexVersion=1479074097480,generation=3,filelist=[_mp.cfe, _mp.cfs, _mp.si, _mq.cfe, _mq.cfs, _mq.si, _mr.cfe, _mr.cfs, _mr.si, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([B6A88B4914AF140A:937F907964E71A09]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([B6A88B4914AF140A:9BD6C413E38BAE06]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/5)={ "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-bfffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"c0000000-ffffffff", "state":"active", "replicas":{}}, "shard3":{ "range":"0-3fffffff", "state":"active", "replicas":{}}, "shard4":{ "range":"40000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/5)={
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-bfffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"0-3fffffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([B6A88B4914AF140A:FEDDFFFD129C3B9F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11514 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/init-core-data-001
[junit4] 2> 740332 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 740333 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001
[junit4] 2> 740333 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 740334 INFO (Thread-5110) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 740334 INFO (Thread-5110) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 740434 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:60131
[junit4] 2> 740456 INFO (jetty-launcher-701-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 740458 INFO (jetty-launcher-701-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 740460 INFO (jetty-launcher-701-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 740462 INFO (jetty-launcher-701-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 740464 INFO (jetty-launcher-701-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f0de7ec{/solr,null,AVAILABLE}
[junit4] 2> 740465 INFO (jetty-launcher-701-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@521e773d{SSL,[ssl, http/1.1]}{127.0.0.1:40417}
[junit4] 2> 740465 INFO (jetty-launcher-701-thread-2) [ ] o.e.j.s.Server Started @744433ms
[junit4] 2> 740466 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40417}
[junit4] 2> 740466 ERROR (jetty-launcher-701-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 740477 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 740477 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 740477 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 740477 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-13T21:16:40.732Z
[junit4] 2> 740470 INFO (jetty-launcher-701-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e82bc48{/solr,null,AVAILABLE}
[junit4] 2> 740469 INFO (jetty-launcher-701-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@753ba8ad{/solr,null,AVAILABLE}
[junit4] 2> 740467 INFO (jetty-launcher-701-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49edbafa{/solr,null,AVAILABLE}
[junit4] 2> 740487 INFO (jetty-launcher-701-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@36bd0e10{SSL,[ssl, http/1.1]}{127.0.0.1:47633}
[junit4] 2> 740487 INFO (jetty-launcher-701-thread-1) [ ] o.e.j.s.Server Started @744455ms
[junit4] 2> 740487 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47633}
[junit4] 2> 740488 ERROR (jetty-launcher-701-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 740488 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 740488 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 740488 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 740488 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-13T21:16:40.743Z
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@1856dc7c{SSL,[ssl, http/1.1]}{127.0.0.1:39538}
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.e.j.s.Server Started @744457ms
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39538}
[junit4] 2> 740490 ERROR (jetty-launcher-701-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 740490 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-13T21:16:40.745Z
[junit4] 2> 740492 INFO (jetty-launcher-701-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@1ee46472{SSL,[ssl, http/1.1]}{127.0.0.1:60969}
[junit4] 2> 740492 INFO (jetty-launcher-701-thread-3) [ ] o.e.j.s.Server Started @744460ms
[junit4] 2> 740492 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60969}
[junit4] 2> 740493 ERROR (jetty-launcher-701-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 740493 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 740493 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 740493 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 740493 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-13T21:16:40.748Z
[junit4] 2> 740499 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 740502 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 740506 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 740512 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 740513 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 740513 INFO (jetty-launcher-701-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60131/solr
[junit4] 2> 740524 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 740526 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 740529 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 740537 INFO (jetty-launcher-701-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60131/solr
[junit4] 2> 740538 INFO (jetty-launcher-701-thread-1) [n:127.0.0.1:47633_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47633_solr
[junit4] 2> 740557 INFO (jetty-launcher-701-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60131/solr
[junit4] 2> 740557 INFO (jetty-launcher-701-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60131/solr
[junit4] 2> 740571 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1585f8c9d0e0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 740573 INFO (jetty-launcher-701-thread-1) [n:127.0.0.1:47633_solr ] o.a.s.c.Overseer Overseer (id=96932449523466246-127.0.0.1:47633_solr-n_0000000000) starting
[junit4] 2> 740602 INFO (jetty-launcher-701-thread-3) [n:127.0.0.1:60969_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60969_solr
[junit4] 2> 740602 INFO (jetty-launcher-701-thread-4) [n:127.0.0.1:39538_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39538_solr
[junit4] 2> 740614 INFO (jetty-launcher-701-thread-1) [n:127.0.0.1:47633_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47633_solr
[junit4] 2> 740619 INFO (zkCallback-710-thread-1-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 740621 INFO (zkCallback-722-thread-2-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 740626 INFO (jetty-launcher-701-thread-2) [n:127.0.0.1:40417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 740629 INFO (OverseerStateUpdate-96932449523466246-127.0.0.1:47633_solr-n_0000000000) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 740633 INFO (jetty-launcher-701-thread-4) [n:127.0.0.1:39538_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node4/.
[junit4] 2> 740636 INFO (zkCallback-721-thread-1-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 740636 INFO (jetty-launcher-701-thread-3) [n:127.0.0.1:60969_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node3/.
[junit4] 2> 740637 INFO (zkCallback-722-thread-1-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 740638 INFO (zkCallback-721-thread-2-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 740639 INFO (jetty-launcher-701-thread-2) [n:127.0.0.1:40417_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40417_solr
[junit4] 2> 740640 INFO (zkCallback-722-thread-1-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 740646 INFO (zkCallback-721-thread-2-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 740667 INFO (zkCallback-723-thread-1-processing-n:127.0.0.1:40417_solr) [n:127.0.0.1:40417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 740667 INFO (jetty-launcher-701-thread-1) [n:127.0.0.1:47633_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node1/.
[junit4] 2> 740675 INFO (zkCallback-710-thread-2-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 740678 INFO (jetty-launcher-701-thread-2) [n:127.0.0.1:40417_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node2/.
[junit4] 2> 740684 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 740731 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 740745 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 740747 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 740760 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_59224_hdfs____fh0aka/webapp
[junit4] 2> 741150 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59224
[junit4] 2> 741214 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 741215 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 741228 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53022_datanode____.g8hipz/webapp
[junit4] 2> 741647 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53022
[junit4] 2> 741680 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 741681 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 741697 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_39851_datanode____.3gndrf/webapp
[junit4] 2> 741762 INFO (IPC Server handler 4 on 56264) [ ] BlockStateChange BLOCK* processReport: from storage DS-c7e0169e-71bd-4ec1-8526-d3277a03a491 node DatanodeRegistration(127.0.0.1:43634, datanodeUuid=837f1d5f-a258-4e15-b631-8abc100e09aa, infoPort=44172, infoSecurePort=0, ipcPort=35246, storageInfo=lv=-56;cid=testClusterID;nsid=1100650242;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 741762 INFO (IPC Server handler 4 on 56264) [ ] BlockStateChange BLOCK* processReport: from storage DS-55ad8a82-6086-42a1-b25e-86b98b667462 node DatanodeRegistration(127.0.0.1:43634, datanodeUuid=837f1d5f-a258-4e15-b631-8abc100e09aa, infoPort=44172, infoSecurePort=0, ipcPort=35246, storageInfo=lv=-56;cid=testClusterID;nsid=1100650242;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 742141 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39851
[junit4] 2> 742283 INFO (IPC Server handler 3 on 56264) [ ] BlockStateChange BLOCK* processReport: from storage DS-cf78a4ba-600b-4c9e-8d1d-5f2080d81c7d node DatanodeRegistration(127.0.0.1:57085, datanodeUuid=d007fb96-3a84-428b-94a7-d966f367e2fa, infoPort=34140, infoSecurePort=0, ipcPort=51957, storageInfo=lv=-56;cid=testClusterID;nsid=1100650242;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 742283 INFO (IPC Server handler 3 on 56264) [ ] BlockStateChange BLOCK* processReport: from storage DS-dce15cc9-5144-4fc7-b359-a7279b3a1f71 node DatanodeRegistration(127.0.0.1:57085, datanodeUuid=d007fb96-3a84-428b-94a7-d966f367e2fa, infoPort=34140, infoSecurePort=0, ipcPort=51957, storageInfo=lv=-56;cid=testClusterID;nsid=1100650242;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 742378 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B6A88B4914AF140A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
[junit4] 2> 742380 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B6A88B4914AF140A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 742442 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 742445 INFO (OverseerThreadFactory-2132-thread-1-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.CreateCollectionCmd Create collection collection
[junit4] 2> 742446 WARN (OverseerThreadFactory-2132-thread-1-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 742446 ERROR (OverseerThreadFactory-2132-thread-1-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 742450 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=7
[junit4] 2> 742452 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B6A88B4914AF140A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
[junit4] 2> 742522 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[B6A88B4914AF140A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCoresAreDistributedAcrossNodes
[junit4] 2> 742523 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[B6A88B4914AF140A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 742526 INFO (qtp1955811460-8826) [n:127.0.0.1:47633_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=nodes_used_collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 742528 INFO (OverseerThreadFactory-2132-thread-2-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.CreateCollectionCmd Create collection nodes_used_collection
[junit4] 2> 742529 INFO (OverseerCollectionConfigSetProcessor-96932449523466246-127.0.0.1:47633_solr-n_0000000000) [n:127.0.0.1:47633_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 742690 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 742721 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 742728 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 742736 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 742840 INFO (zkCallback-722-thread-1-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 742840 INFO (zkCallback-710-thread-4-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 742840 INFO (zkCallback-723-thread-1-processing-n:127.0.0.1:40417_solr) [n:127.0.0.1:40417_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 742841 INFO (zkCallback-721-thread-2-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 743706 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 743715 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica1] Schema name=minimal
[junit4] 2> 743718 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 743719 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 743727 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56264/data
[junit4] 2> 743727 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 743727 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node1/nodes_used_collection_shard1_replica1], dataDir=[hdfs://localhost:56264/data/nodes_used_collection/core_node1/data/]
[junit4] 2> 743727 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node1/data/snapshot_metadata
[junit4] 2> 743737 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743737 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743737 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743744 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 743744 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 743749 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 743754 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica2] Schema name=minimal
[junit4] 2> 743754 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica2] Schema name=minimal
[junit4] 2> 743771 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 743773 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica1] Schema name=minimal
[junit4] 2> 743773 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 743776 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 743779 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743780 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node1/data
[junit4] 2> 743780 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 743781 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 743781 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56264/data
[junit4] 2> 743782 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 743782 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node2/nodes_used_collection_shard2_replica1], dataDir=[hdfs://localhost:56264/data/nodes_used_collection/core_node4/data/]
[junit4] 2> 743782 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node4/data/snapshot_metadata
[junit4] 2> 743786 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56264/data
[junit4] 2> 743786 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 743786 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node4/nodes_used_collection_shard1_replica2], dataDir=[hdfs://localhost:56264/data/nodes_used_collection/core_node3/data/]
[junit4] 2> 743786 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node3/data/snapshot_metadata
[junit4] 2> 743788 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743788 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743788 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743778 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 743790 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56264/data
[junit4] 2> 743790 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 743790 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B6A88B4914AF140A-001/tempDir-001/node3/nodes_used_collection_shard2_replica2], dataDir=[hdfs://localhost:56264/data/nodes_used_collection/core_node2/data/]
[junit4] 2> 743792 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node2/data/snapshot_metadata
[junit4] 2> 743797 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743798 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743798 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743798 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743803 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node4/data
[junit4] 2> 743805 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node1/data/index
[junit4] 2> 743810 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743811 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node3/data
[junit4] 2> 743813 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743813 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743813 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743818 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743819 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743819 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743827 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743828 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node2/data
[junit4] 2> 743834 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743835 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node3/data/index
[junit4] 2> 743837 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node4/data/index
[junit4] 2> 743843 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743844 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743844 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743846 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743846 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743846 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743852 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56264/data/nodes_used_collection/core_node2/data/index
[junit4] 2> 743871 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 743872 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 743872 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 743875 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743875 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743904 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 743908 INFO (IPC Server handler 0 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57085 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c7e0169e-71bd-4ec1-8526-d3277a03a491:NORMAL:127.0.0.1:43634|RBW], ReplicaUC[[DISK]DS-cf78a4ba-600b-4c9e-8d1d-5f2080d81c7d:NORMAL:127.0.0.1:57085|FINALIZED]]} size 0
[junit4] 2> 743909 INFO (IPC Server handler 3 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43634 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-cf78a4ba-600b-4c9e-8d1d-5f2080d81c7d:NORMAL:127.0.0.1:57085|FINALIZED], ReplicaUC[[DISK]DS-55ad8a82-6086-42a1-b25e-86b98b667462:NORMAL:127.0.0.1:43634|FINALIZED]]} size 0
[junit4] 2> 743917 INFO (IPC Server handler 1 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57085 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-55ad8a82-6086-42a1-b25e-86b98b667462:NORMAL:127.0.0.1:43634|RBW], ReplicaUC[[DISK]DS-dce15cc9-5144-4fc7-b359-a7279b3a1f71:NORMAL:127.0.0.1:57085|FINALIZED]]} size 0
[junit4] 2> 743919 INFO (IPC Server handler 2 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43634 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-dce15cc9-5144-4fc7-b359-a7279b3a1f71:NORMAL:127.0.0.1:57085|FINALIZED], ReplicaUC[[DISK]DS-c7e0169e-71bd-4ec1-8526-d3277a03a491:NORMAL:127.0.0.1:43634|FINALIZED]]} size 0
[junit4] 2> 743942 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 743942 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 743942 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 743946 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 743946 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 743946 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 743952 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 743952 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 743956 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 743956 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 743960 INFO (IPC Server handler 1 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43634 is added to blk_1073741828_1004{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-55ad8a82-6086-42a1-b25e-86b98b667462:NORMAL:127.0.0.1:43634|RBW], ReplicaUC[[DISK]DS-dce15cc9-5144-4fc7-b359-a7279b3a1f71:NORMAL:127.0.0.1:57085|RBW]]} size 71
[junit4] 2> 743960 INFO (IPC Server handler 9 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57085 is added to blk_1073741828_1004 size 71
[junit4] 2> 743961 INFO (IPC Server handler 2 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57085 is added to blk_1073741827_1003{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-55ad8a82-6086-42a1-b25e-86b98b667462:NORMAL:127.0.0.1:43634|RBW], ReplicaUC[[DISK]DS-cf78a4ba-600b-4c9e-8d1d-5f2080d81c7d:NORMAL:127.0.0.1:57085|RBW]]} size 71
[junit4] 2> 743962 INFO (IPC Server handler 3 on 56264) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43634 is added to blk_1073741827_1003 size 71
[junit4] 2> 743971 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15675385[nodes_used_collection_shard1_replica1] main]
[junit4] 2> 743975 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 743975 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 743975 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 743975 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550919196192276480
[junit4] 2> 743977 INFO (searcherExecutor-2141-thread-1-processing-n:127.0.0.1:47633_solr x:nodes_used_collection_shard1_replica1 s:shard1 c:nodes_used_collection r:core_node1) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@15675385[nodes_used_collection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 743983 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 743990 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b8066ab[nodes_used_collection_shard2_replica1] main]
[junit4] 2> 743991 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 743992 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 743992 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 743992 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550919196210102272
[junit4] 2> 743994 INFO (searcherExecutor-2142-thread-1-processing-n:127.0.0.1:40417_solr x:nodes_used_collection_shard2_replica1 s:shard2 c:nodes_used_collection r:core_node4) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@7b8066ab[nodes_used_collection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 743999 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 744102 INFO (zkCallback-722-thread-1-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744102 INFO (zkCallback-723-thread-1-processing-n:127.0.0.1:40417_solr) [n:127.0.0.1:40417_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744102 INFO (zkCallback-721-thread-2-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744102 INFO (zkCallback-710-thread-1-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744374 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 744374 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 744374 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 744379 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 744379 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 744379 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 744383 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 744384 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 744392 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5720ca5c[nodes_used_collection_shard2_replica2] main]
[junit4] 2> 744393 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 744393 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 744394 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 744394 INFO (qtp50170523-8844) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550919196631629824
[junit4] 2> 744396 INFO (searcherExecutor-2144-thread-1-processing-n:127.0.0.1:60969_solr x:nodes_used_collection_shard2_replica2 s:shard2 c:nodes_used_collection r:core_node2) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@5720ca5c[nodes_used_collection_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 744400 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 744400 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 744413 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@8ebbd8c[nodes_used_collection_shard1_replica2] main]
[junit4] 2> 744414 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 744414 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 744415 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 744415 INFO (searcherExecutor-2143-thread-1-processing-n:127.0.0.1:39538_solr x:nodes_used_collection_shard1_replica2 s:shard1 c:nodes_used_collection r:core_node3) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@8ebbd8c[nodes_used_collection_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 744415 INFO (qtp618199129-8834) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550919196653649920
[junit4] 2> 744483 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 744483 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 744483 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:47633/solr/nodes_used_collection_shard1_replica1/
[junit4] 2> 744484 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard1_replica1 url=https://127.0.0.1:47633/solr START replicas=[https://127.0.0.1:39538/solr/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4] 2> 744484 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard1_replica1 url=https://127.0.0.1:47633/solr DONE. We have no versions. sync failed.
[junit4] 2> 744489 INFO (qtp618199129-8832) [n:127.0.0.1:39538_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.S.Request [nodes_used_collection_shard1_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 744490 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 744490 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 744494 INFO (qtp1955811460-8825) [n:127.0.0.1:47633_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:47633/solr/nodes_used_collection_shard1_replica1/ shard1
[junit4] 2> 744499 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 744499 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 744499 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40417/solr/nodes_used_collection_shard2_replica1/
[junit4] 2> 744499 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard2_replica1 url=https://127.0.0.1:40417/solr START replicas=[https://127.0.0.1:60969/solr/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4] 2> 744500 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard2_replica1 url=https://127.0.0.1:40417/solr DONE. We have no versions. sync failed.
[junit4] 2> 744508 INFO (qtp50170523-8841) [n:127.0.0.1:60969_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.S.Request [nodes_used_collection_shard2_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 744509 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 744509 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 744513 INFO (qtp854121126-8815) [n:127.0.0.1:40417_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40417/solr/nodes_used_collection_shard2_replica1/ shard2
[junit4] 2> 744616 INFO (zkCallback-722-thread-1-processing-n:127.0.0.1:39538_solr) [n:127.0.0.1:39538_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744616 INFO (zkCallback-710-thread-2-processing-n:127.0.0.1:47633_solr) [n:127.0.0.1:47633_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744616 INFO (zkCallback-721-thread-2-processing-n:127.0.0.1:60969_solr) [n:127.0.0.1:60969_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 744616 INFO (zkCallback-723-thread-1-processing-n:127.0.0.1:40417_solr) [n:127.0.0.1:40417_solr ] o.a.s.c.c.ZkSta
[...truncated too long message...]
ChaosMonkey monkey: stop shard! 54880
[junit4] 2> 4707760 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1629447914
[junit4] 2> 4707767 INFO (coreCloseExecutor-16385-thread-1) [n:127.0.0.1:54880_dxgi%2Fa c:target_collection s:shard2 r:core_node4 x:target_collection_shard2_replica1] o.a.s.c.SolrCore [target_collection_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@7e55d4f9
[junit4] 2> 4707767 INFO (coreCloseExecutor-16385-thread-1) [n:127.0.0.1:54880_dxgi%2Fa c:target_collection s:shard2 r:core_node4 x:target_collection_shard2_replica1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ target_collection:shard2
[junit4] 2> 4707780 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.a.s.c.Overseer Overseer (id=96932688936501281-127.0.0.1:54880_dxgi%2Fa-n_0000000007) closing
[junit4] 2> 4707780 INFO (OverseerStateUpdate-96932688936501281-127.0.0.1:54880_dxgi%2Fa-n_0000000007) [n:127.0.0.1:54880_dxgi%2Fa ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:54880_dxgi%2Fa
[junit4] 2> 4709282 WARN (zkCallback-37223-thread-5-processing-n:127.0.0.1:54880_dxgi%2Fa) [n:127.0.0.1:54880_dxgi%2Fa ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4709283 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@4a0f507d{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 4709283 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3b42fcec{/dxgi/a,null,UNAVAILABLE}
[junit4] 2> 4709284 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReplicationAfterRestart
[junit4] 2> 4709285 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationAfterRestart-seed#[B6A88B4914AF140A]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53265 53265
[junit4] 2> 4709467 INFO (Thread-15974) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:53265 53265
[junit4] 2> 4709468 WARN (Thread-15974) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 35 /solr/aliases.json
[junit4] 2> 18 /solr/configs/conf1
[junit4] 2> 10 /solr/security.json
[junit4] 2> 8 /solr/collections/tmp_collection/state.json
[junit4] 2> 5 /solr/collections/source_collection/leaders/shard1/leader
[junit4] 2> 2 /solr/collections/source_collection/leaders/shard2/leader
[junit4] 2> 2 /solr/collections/tmp_collection/leaders/shard2/leader
[junit4] 2> 2 /solr/collections/target_collection/leaders/shard2/leader
[junit4] 2> 2 /solr/collections/tmp_collection/leaders/shard1/leader
[junit4] 2> 2 /solr/collections/target_collection/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 35 /solr/clusterprops.json
[junit4] 2> 35 /solr/clusterstate.json
[junit4] 2> 9 /solr/collections/source_collection/cdcr/state/process
[junit4] 2> 8 /solr/collections/tmp_collection/cdcr/state/process
[junit4] 2> 8 /solr/collections/source_collection/cdcr/state/buffer
[junit4] 2> 8 /solr/collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 6 /solr/collections/source_collection/state.json
[junit4] 2> 4 /solr/collections/target_collection/cdcr/state/buffer
[junit4] 2> 4 /solr/collections/target_collection/cdcr/state/process
[junit4] 2> 4 /solr/collections/target_collection/state.json
[junit4] 2> 2 /solr/collections/source_collection/leader_elect/shard1/election/96932688936501259-core_node4-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/96932688936501281-127.0.0.1:54880_dxgi%2Fa-n_0000000007
[junit4] 2> 2 /solr/overseer_elect/election/96932688936501256-127.0.0.1:43664_dxgi%2Fa-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 300 /solr/overseer/collection-queue-work
[junit4] 2> 82 /solr/overseer/queue
[junit4] 2> 35 /solr/collections
[junit4] 2> 33 /solr/live_nodes
[junit4] 2> 25 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 4709469 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[B6A88B4914AF140A]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CdcrReplicationDistributedZkTest_B6A88B4914AF140A-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=no-NO, timezone=Pacific/Pohnpei
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=160718320,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [StatelessScriptUpdateProcessorFactoryTest, SpellPossibilityIteratorTest, TestPostingsSolrHighlighter, BlockJoinFacetSimpleTest, DistributedQueueTest, PreAnalyzedFieldTest, TestZkChroot, TestUseDocValuesAsStored2, TestAnalyzedSuggestions, TestSolrDeletionPolicy1, TestCloudRecovery, TestReRankQParserPlugin, TestSolr4Spatial2, DebugComponentTest, TestLFUCache, RequestLoggingTest, FieldMutatingUpdateProcessorTest, CoreAdminHandlerTest, ShowFileRequestHandlerTest, TestSolrConfigHandlerCloud, TlogReplayBufferedWhileIndexingTest, BadComponentTest, DOMUtilTest, CollectionsAPISolrJTest, TestHighFrequencyDictionaryFactory, DistanceUnitsTest, TestTolerantSearch, TimeZoneUtilsTest, LukeRequestHandlerTest, HttpPartitionTest, BigEndianAscendingWordDeserializerTest, PluginInfoTest, DistributedQueryComponentOptimizationTest, TestIndexingPerformance, TestCollapseQParserPlugin, TestCloudManagedSchema, DirectUpdateHandlerOptimizeTest, BlobRepositoryCloudTest, SolrTestCaseJ4Test, TestClusterStateMutator, MinimalSchemaTest, DistribCursorPagingTest, TestRecovery, LeaderElectionTest, TestLocalFSCloudBackupRestore, TestReload, ConnectionReuseTest, SOLR749Test, TestConfig, LeaderFailureAfterFreshStartTest, TestScoreJoinQPScore, CircularListTest, CdcrRequestHandlerTest, TestReplicaProperties, TestStressLiveNodes, TestExactSharedStatsCache, BadIndexSchemaTest, XmlUpdateRequestHandlerTest, OpenExchangeRatesOrgProviderTest, TestManagedSynonymFilterFactory, TestPushWriter, TestMacroExpander, AssignTest, WrapperMergePolicyFactoryTest, TestDFRSimilarityFactory, ExplicitHLLTest, TestWriterPerf, TestCryptoKeys, OverseerRolesTest, SecurityConfHandlerTest, TestSchemaNameResource, TestFieldSortValues, TestNumericTerms32, BasicAuthIntegrationTest, RollingRestartTest, ReplaceNodeTest, RecoveryZkTest, SuggesterWFSTTest, PrimUtilsTest, CdcrBootstrapTest, FieldAnalysisRequestHandlerTest, DataDrivenBlockJoinTest, HdfsDirectoryFactoryTest, DirectUpdateHandlerTest, TestSolrXml, PingRequestHandlerTest, NoCacheHeaderTest, SuggesterTSTTest, UpdateParamsTest, TestSolrCloudWithDelegationTokens, TestCollationField, TestNonDefinedSimilarityFactory, CleanupOldIndexTest, TestBinaryField, HdfsUnloadDistributedZkTest, HdfsChaosMonkeySafeLeaderTest, TestBlendedInfixSuggestions, CollectionStateFormat2Test, RequestHandlersTest, TestDocTermOrds, DeleteReplicaTest, TestIndexSearcher, DocValuesMultiTest, TermVectorComponentDistributedTest, MigrateRouteKeyTest, AutoCommitTest, TestDistributedGrouping, TestHashPartitioner, DistributedSpellCheckComponentTest, TestStressReorder, TestJoin, HardAutoCommitTest, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, TestSort, TestFunctionQuery, TestLazyCores, TestBadConfig, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SolrRequestParserTest, DocValuesTest, SuggesterTest, TestCSVLoader, PolyFieldTest, SolrCoreCheckLockOnStartupTest, TestUpdate, QueryEqualityTest, SortByFunctionTest, TestRemoteStreaming, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, TestOmitPositions, FileBasedSpellCheckerTest, TermsComponentTest, DocumentBuilderTest, TestValueSourceCache, TermVectorComponentTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, JSONWriterTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestQuerySenderListener, AlternateDirectoryTest, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, TestMergePolicyConfig, TestSearchPerf, NumericFieldsTest, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, TestLMJelinekMercerSimilarityFactory, TestJmxMonitoredMap, URLClassifyProcessorTest, ClusterStateTest, TestDocumentBuilder, UUIDFieldTest, DistributedMLTComponentTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, AsyncCallRequestStatusResponseTest, CdcrReplicationDistributedZkTest]
[junit4] Completed [527/651 (3!)] on J2 in 1774.99s, 11 tests, 1 failure <<< FAILURES!
[...truncated 61020 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 197 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/197/
2 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/9)={ "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-bfffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"c0000000-ffffffff", "state":"active", "replicas":{}}, "shard3":{ "range":"0-3fffffff", "state":"active", "replicas":{}}, "shard4":{ "range":"40000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"3", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/9)={
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-bfffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"0-3fffffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"3",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([F271390D7968BA19:BA044DB97F5B958C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([F271390D7968BA19:DF0F76578E4C0015]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12882 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/init-core-data-001
[junit4] 2> 5101576 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 5101578 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001
[junit4] 2> 5101578 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 5101578 INFO (Thread-65840) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 5101578 INFO (Thread-65840) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 5101678 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:59209
[junit4] 2> 5101687 INFO (jetty-launcher-4801-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 5101688 INFO (jetty-launcher-4801-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 5101690 INFO (jetty-launcher-4801-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 5101702 INFO (jetty-launcher-4801-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 5101706 INFO (jetty-launcher-4801-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6500f29e{/solr,null,AVAILABLE}
[junit4] 2> 5101712 INFO (jetty-launcher-4801-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2f4f4639{/solr,null,AVAILABLE}
[junit4] 2> 5101714 INFO (jetty-launcher-4801-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@3a19ead3{SSL,[ssl, http/1.1]}{127.0.0.1:51302}
[junit4] 2> 5101714 INFO (jetty-launcher-4801-thread-1) [ ] o.e.j.s.Server Started @5108842ms
[junit4] 2> 5101714 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51302}
[junit4] 2> 5101714 ERROR (jetty-launcher-4801-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5101714 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 5101714 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5101715 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5101715 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-12T21:21:45.876Z
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@50a67e8{SSL,[ssl, http/1.1]}{127.0.0.1:46043}
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.e.j.s.Server Started @5108846ms
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46043}
[junit4] 2> 5101718 ERROR (jetty-launcher-4801-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5101718 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-12T21:21:45.879Z
[junit4] 2> 5101724 INFO (jetty-launcher-4801-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d9a3fae{/solr,null,AVAILABLE}
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@6ed80bb8{SSL,[ssl, http/1.1]}{127.0.0.1:32880}
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.e.j.s.Server Started @5108854ms
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32880}
[junit4] 2> 5101726 ERROR (jetty-launcher-4801-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5101726 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-12T21:21:45.887Z
[junit4] 2> 5101728 INFO (jetty-launcher-4801-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a771bc1{/solr,null,AVAILABLE}
[junit4] 2> 5101729 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@7fbd0882{SSL,[ssl, http/1.1]}{127.0.0.1:50382}
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.e.j.s.Server Started @5108858ms
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50382}
[junit4] 2> 5101730 ERROR (jetty-launcher-4801-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5101730 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-12T21:21:45.891Z
[junit4] 2> 5101741 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5101742 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5101747 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5101748 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 5101749 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 5101751 INFO (jetty-launcher-4801-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59209/solr
[junit4] 2> 5101752 INFO (jetty-launcher-4801-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59209/solr
[junit4] 2> 5101753 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 5101754 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 5101754 INFO (jetty-launcher-4801-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59209/solr
[junit4] 2> 5101755 INFO (jetty-launcher-4801-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59209/solr
[junit4] 2> 5101758 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1585a6ae90c0009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 5101759 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1585a6ae90c0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 5101778 INFO (jetty-launcher-4801-thread-2) [n:127.0.0.1:32880_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32880_solr
[junit4] 2> 5101781 INFO (jetty-launcher-4801-thread-2) [n:127.0.0.1:32880_solr ] o.a.s.c.Overseer Overseer (id=96926807211376648-127.0.0.1:32880_solr-n_0000000000) starting
[junit4] 2> 5101797 INFO (jetty-launcher-4801-thread-3) [n:127.0.0.1:46043_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46043_solr
[junit4] 2> 5101797 INFO (jetty-launcher-4801-thread-1) [n:127.0.0.1:51302_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51302_solr
[junit4] 2> 5101805 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5101805 INFO (zkCallback-4822-thread-1-processing-n:127.0.0.1:51302_solr) [n:127.0.0.1:51302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5101805 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 5101806 INFO (jetty-launcher-4801-thread-4) [n:127.0.0.1:50382_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50382_solr
[junit4] 2> 5101808 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 5101809 INFO (OverseerStateUpdate-96926807211376648-127.0.0.1:32880_solr-n_0000000000) [n:127.0.0.1:32880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5101811 INFO (jetty-launcher-4801-thread-2) [n:127.0.0.1:32880_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32880_solr
[junit4] 2> 5101812 INFO (jetty-launcher-4801-thread-3) [n:127.0.0.1:46043_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node3/.
[junit4] 2> 5101812 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5101813 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5101813 INFO (zkCallback-4819-thread-2-processing-n:127.0.0.1:32880_solr) [n:127.0.0.1:32880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5101813 INFO (zkCallback-4822-thread-1-processing-n:127.0.0.1:51302_solr) [n:127.0.0.1:51302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5101814 INFO (zkCallback-4819-thread-1-processing-n:127.0.0.1:32880_solr) [n:127.0.0.1:32880_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 5101815 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 5101815 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 5101821 INFO (jetty-launcher-4801-thread-4) [n:127.0.0.1:50382_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node4/.
[junit4] 2> 5101822 INFO (zkCallback-4822-thread-1-processing-n:127.0.0.1:51302_solr) [n:127.0.0.1:51302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 5101828 INFO (jetty-launcher-4801-thread-2) [n:127.0.0.1:32880_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node2/.
[junit4] 2> 5101837 INFO (jetty-launcher-4801-thread-1) [n:127.0.0.1:51302_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node1/.
[junit4] 2> 5101843 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 5101899 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 5101904 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 5101906 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 5101919 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_38756_hdfs____i0cldp/webapp
[junit4] 2> 5102279 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38756
[junit4] 2> 5102333 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 5102334 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 5102348 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_49498_datanode____.65pfmz/webapp
[junit4] 2> 5102762 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49498
[junit4] 2> 5102844 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 5102846 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 5102862 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33153_datanode____5122za/webapp
[junit4] 2> 5102952 INFO (IPC Server handler 3 on 45374) [ ] BlockStateChange BLOCK* processReport: from storage DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993 node DatanodeRegistration(127.0.0.1:40559, datanodeUuid=4dbe593a-b7bb-4a45-9565-5734b076e4d6, infoPort=51325, infoSecurePort=0, ipcPort=35591, storageInfo=lv=-56;cid=testClusterID;nsid=1705878909;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 5102952 INFO (IPC Server handler 3 on 45374) [ ] BlockStateChange BLOCK* processReport: from storage DS-bc3221f5-a4c1-48ad-8fb7-0e99d2db95b3 node DatanodeRegistration(127.0.0.1:40559, datanodeUuid=4dbe593a-b7bb-4a45-9565-5734b076e4d6, infoPort=51325, infoSecurePort=0, ipcPort=35591, storageInfo=lv=-56;cid=testClusterID;nsid=1705878909;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 5103256 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33153
[junit4] 2> 5103408 INFO (IPC Server handler 3 on 45374) [ ] BlockStateChange BLOCK* processReport: from storage DS-2de90e67-b58c-488a-88cf-519b872ff0cb node DatanodeRegistration(127.0.0.1:43585, datanodeUuid=382b1db1-0ad9-4e4b-ae0a-4957faf49078, infoPort=38492, infoSecurePort=0, ipcPort=55996, storageInfo=lv=-56;cid=testClusterID;nsid=1705878909;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 5103408 INFO (IPC Server handler 3 on 45374) [ ] BlockStateChange BLOCK* processReport: from storage DS-a1bb6490-d10f-40bb-8c4e-1cacfe927458 node DatanodeRegistration(127.0.0.1:43585, datanodeUuid=382b1db1-0ad9-4e4b-ae0a-4957faf49078, infoPort=38492, infoSecurePort=0, ipcPort=55996, storageInfo=lv=-56;cid=testClusterID;nsid=1705878909;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 5103587 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[F271390D7968BA19]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadActionNames
[junit4] 2> 5103588 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[F271390D7968BA19]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 5103655 ERROR (qtp606807415-85466) [n:127.0.0.1:50382_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown action: BADACTION
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:203)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 5103656 INFO (qtp606807415-85466) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=badactioncollection&action=BADACTION&numShards=2&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 5103656 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[F271390D7968BA19]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadActionNames
[junit4] 2> 5103728 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[F271390D7968BA19]) [ ] o.a.s.SolrTestCaseJ4 ###Starting deletePartiallyCreatedCollection
[junit4] 2> 5103729 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[F271390D7968BA19]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 5103734 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2
[junit4] 2> 5103735 INFO (OverseerStateUpdate-96926807211376648-127.0.0.1:32880_solr-n_0000000000) [n:127.0.0.1:32880_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 5104746 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 5104754 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
[junit4] 2> 5104757 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5104758 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection
[junit4] 2> 5104758 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45374/data
[junit4] 2> 5104759 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 5104759 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node4/halfdeletedcollection_shard1_replica1], dataDir=[hdfs://localhost:45374/data/halfdeletedcollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003/]
[junit4] 2> 5104759 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003/snapshot_metadata
[junit4] 2> 5104766 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5104766 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5104766 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5104771 WARN (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 5109775 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5109776 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003
[junit4] 2> 5109796 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003/index
[junit4] 2> 5109803 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5109803 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5109803 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5109811 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5109841 INFO (IPC Server handler 6 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43585 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bc3221f5-a4c1-48ad-8fb7-0e99d2db95b3:NORMAL:127.0.0.1:40559|RBW], ReplicaUC[[DISK]DS-a1bb6490-d10f-40bb-8c4e-1cacfe927458:NORMAL:127.0.0.1:43585|RBW]]} size 0
[junit4] 2> 5109841 INFO (IPC Server handler 0 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40559 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bc3221f5-a4c1-48ad-8fb7-0e99d2db95b3:NORMAL:127.0.0.1:40559|RBW], ReplicaUC[[DISK]DS-a1bb6490-d10f-40bb-8c4e-1cacfe927458:NORMAL:127.0.0.1:43585|RBW]]} size 0
[junit4] 2> 5109862 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 5109862 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5109862 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 5109875 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5109875 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5109883 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7ca61d0b[halfdeletedcollection_shard1_replica1] main]
[junit4] 2> 5109884 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 5109885 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 5109885 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 5109885 INFO (searcherExecutor-10345-thread-1-processing-n:127.0.0.1:50382_solr x:halfdeletedcollection_shard1_replica1 s:shard2 c:halfdeletedcollection r:core_node1) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@7ca61d0b[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5109885 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550828924091498496
[junit4] 2> 5109893 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5109893 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5109893 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/
[junit4] 2> 5109896 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5109896 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/ has no replicas
[junit4] 2> 5109902 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/ shard2
[junit4] 2> 5110053 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5110057 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard2 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003&collection.configName=conf&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&wt=javabin&version=2} status=0 QTime=6324
[junit4] 2> 5110059 INFO (qtp606807415-85458) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5110076 INFO (OverseerThreadFactory-10336-thread-1-processing-n:127.0.0.1:32880_solr) [n:127.0.0.1:32880_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 5110085 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@27f2cf1d
[junit4] 2> 5110090 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003/index
[junit4] 2> 5110090 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003/snapshot_metadata
[junit4] 2> 5110091 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-003
[junit4] 2> 5110099 INFO (IPC Server handler 1 on 45374) [ ] BlockStateChange BLOCK* addToInvalidates: blk_1073741825_1001 127.0.0.1:40559 127.0.0.1:43585
[junit4] 2> 5110101 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 5110807 INFO (qtp606807415-85458) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=748
[junit4] 2> 5110809 INFO (qtp606807415-85464) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5110809 INFO (qtp606807415-85464) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5110811 INFO (qtp606807415-85464) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5110816 INFO (OverseerThreadFactory-10336-thread-2-processing-n:127.0.0.1:32880_solr) [n:127.0.0.1:32880_solr ] o.a.s.c.CreateCollectionCmd Create collection halfdeletedcollection
[junit4] 2> 5110816 INFO (OverseerCollectionConfigSetProcessor-96926807211376648-127.0.0.1:32880_solr-n_0000000000) [n:127.0.0.1:32880_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5111026 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2
[junit4] 2> 5111079 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2
[junit4] 2> 5111184 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5111185 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5111320 INFO (org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5b2a5718) [ ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:40559 to delete [blk_1073741825_1001]
[junit4] 2> 5112042 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 5112052 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard1_replica1] Schema name=minimal
[junit4] 2> 5112059 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5112061 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard1_replica1' using configuration from collection halfdeletedcollection
[junit4] 2> 5112061 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45374/data
[junit4] 2> 5112062 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 5112062 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node4/halfdeletedcollection_shard1_replica1], dataDir=[hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data/]
[junit4] 2> 5112062 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data/snapshot_metadata
[junit4] 2> 5112074 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5112074 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5112074 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5112088 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5112089 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data
[junit4] 2> 5112090 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 5112099 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema [halfdeletedcollection_shard2_replica1] Schema name=minimal
[junit4] 2> 5112102 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5112103 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfdeletedcollection_shard2_replica1' using configuration from collection halfdeletedcollection
[junit4] 2> 5112103 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45374/data
[junit4] 2> 5112103 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 5112103 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [[halfdeletedcollection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node3/halfdeletedcollection_shard2_replica1], dataDir=[hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data/]
[junit4] 2> 5112103 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data/snapshot_metadata
[junit4] 2> 5112127 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data/index
[junit4] 2> 5112131 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5112131 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5112131 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5112139 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5112140 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5112140 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5112145 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5112149 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data
[junit4] 2> 5112152 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5112176 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data/index
[junit4] 2> 5112179 INFO (IPC Server handler 2 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40559 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-2de90e67-b58c-488a-88cf-519b872ff0cb:NORMAL:127.0.0.1:43585|RBW], ReplicaUC[[DISK]DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993:NORMAL:127.0.0.1:40559|RBW]]} size 0
[junit4] 2> 5112180 INFO (IPC Server handler 0 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43585 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-2de90e67-b58c-488a-88cf-519b872ff0cb:NORMAL:127.0.0.1:43585|RBW], ReplicaUC[[DISK]DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993:NORMAL:127.0.0.1:40559|RBW]]} size 0
[junit4] 2> 5112186 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5112186 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5112186 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5112197 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 5112197 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5112197 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 5112200 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5112208 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5112208 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5112216 INFO (IPC Server handler 0 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43585 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993:NORMAL:127.0.0.1:40559|RBW], ReplicaUC[[DISK]DS-a1bb6490-d10f-40bb-8c4e-1cacfe927458:NORMAL:127.0.0.1:43585|FINALIZED]]} size 0
[junit4] 2> 5112216 INFO (IPC Server handler 6 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40559 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a1bb6490-d10f-40bb-8c4e-1cacfe927458:NORMAL:127.0.0.1:43585|FINALIZED], ReplicaUC[[DISK]DS-bc3221f5-a4c1-48ad-8fb7-0e99d2db95b3:NORMAL:127.0.0.1:40559|FINALIZED]]} size 0
[junit4] 2> 5112218 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@742bb306[halfdeletedcollection_shard1_replica1] main]
[junit4] 2> 5112219 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 5112220 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 5112220 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 5112220 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550828926539923456
[junit4] 2> 5112222 INFO (searcherExecutor-10350-thread-1-processing-n:127.0.0.1:50382_solr x:halfdeletedcollection_shard1_replica1 s:shard1 c:halfdeletedcollection r:core_node1) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] Registered new searcher Searcher@742bb306[halfdeletedcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5112229 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5112230 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5112230 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/
[junit4] 2> 5112230 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5112230 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/ has no replicas
[junit4] 2> 5112230 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 5112230 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5112230 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 5112235 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50382/solr/halfdeletedcollection_shard1_replica1/ shard1
[junit4] 2> 5112242 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5112242 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5112249 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@44137b59[halfdeletedcollection_shard2_replica1] main]
[junit4] 2> 5112250 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 5112251 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 5112251 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 5112251 INFO (searcherExecutor-10351-thread-1-processing-n:127.0.0.1:46043_solr x:halfdeletedcollection_shard2_replica1 s:shard2 c:halfdeletedcollection r:core_node2) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1] Registered new searcher Searcher@44137b59[halfdeletedcollection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5112251 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550828926572429312
[junit4] 2> 5112258 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 5112258 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 5112258 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:46043/solr/halfdeletedcollection_shard2_replica1/
[junit4] 2> 5112258 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 5112258 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:46043/solr/halfdeletedcollection_shard2_replica1/ has no replicas
[junit4] 2> 5112262 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:46043/solr/halfdeletedcollection_shard2_replica1/ shard2
[junit4] 2> 5112366 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5112366 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5112386 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5112387 INFO (qtp606807415-85463) [n:127.0.0.1:50382_solr c:halfdeletedcollection s:shard1 r:core_node1 x:halfdeletedcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard1&wt=javabin&version=2} status=0 QTime=1361
[junit4] 2> 5112415 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 5112418 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfdeletedcollection s:shard2 r:core_node2 x:halfdeletedcollection_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfdeletedcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfdeletedcollection&shard=shard2&wt=javabin&version=2} status=0 QTime=1338
[junit4] 2> 5112420 INFO (qtp606807415-85464) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 5112519 INFO (zkCallback-4821-thread-1-processing-n:127.0.0.1:50382_solr) [n:127.0.0.1:50382_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5112519 INFO (zkCallback-4820-thread-1-processing-n:127.0.0.1:46043_solr) [n:127.0.0.1:46043_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfdeletedcollection/state.json] for collection [halfdeletedcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 5112818 INFO (OverseerCollectionConfigSetProcessor-96926807211376648-127.0.0.1:32880_solr-n_0000000000) [n:127.0.0.1:32880_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 5113420 INFO (qtp606807415-85464) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfdeletedcollection&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2609
[junit4] 2> 5113422 INFO (qtp606807415-85466) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5113423 INFO (qtp606807415-85466) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 5113423 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.deletePartiallyCreatedCollection-seed#[F271390D7968BA19]) [ ] o.a.s.SolrTestCaseJ4 ###Ending deletePartiallyCreatedCollection
[junit4] 2> 5113516 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[F271390D7968BA19]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateShouldFailOnExistingCore
[junit4] 2> 5113517 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[F271390D7968BA19]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 5113521 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfdeletedcollection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 5113524 INFO (OverseerThreadFactory-10336-thread-3-processing-n:127.0.0.1:32880_solr) [n:127.0.0.1:32880_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 5113527 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.c.SolrCore [halfdeletedcollection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@71ce9bd5
[junit4] 2> 5113529 INFO (qtp1409536826-85447) [n:127.0.0.1:46043_solr ] o.a.s.c.SolrCore [halfdeletedcollection_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@3f4b06ef
[junit4] 2> 5113534 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data/index
[junit4] 2> 5113535 INFO (qtp1409536826-85447) [n:127.0.0.1:46043_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data/snapshot_metadata
[junit4] 2> 5113535 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data/snapshot_metadata
[junit4] 2> 5113535 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node1/data
[junit4] 2> 5113540 INFO (qtp1409536826-85447) [n:127.0.0.1:46043_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data/index
[junit4] 2> 5113540 INFO (qtp1409536826-85447) [n:127.0.0.1:46043_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:45374/data/halfdeletedcollection/core_node2/data
[junit4] 2> 5113543 INFO (IPC Server handler 5 on 45374) [ ] BlockStateChange BLOCK* addToInvalidates: blk_1073741826_1002 127.0.0.1:43585 127.0.0.1:40559
[junit4] 2> 5113547 INFO (qtp606807415-85467) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 5113553 INFO (IPC Server handler 9 on 45374) [ ] BlockStateChange BLOCK* addToInvalidates: blk_1073741827_1003 127.0.0.1:40559 127.0.0.1:43585
[junit4] 2> 5113557 INFO (qtp1409536826-85447) [n:127.0.0.1:46043_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfdeletedcollection_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=29
[junit4] 2> 5114265 INFO (qtp606807415-85465) [n:127.0.0.1:50382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=halfdeletedcollection&action=DELETE&wt=javabin&version=2} status=0 QTime=743
[junit4] 2> 5114273 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-004&collection.configName=conf&name=halfcollection_shard1_replica1&action=CREATE&numShards=1&collection=halfcollectionblocker&wt=javabin&version=2
[junit4] 2> 5114275 INFO (OverseerStateUpdate-96926807211376648-127.0.0.1:32880_solr-n_0000000000) [n:127.0.0.1:32880_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 5114320 INFO (org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5b2a5718) [ ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:43585 to delete [blk_1073741825_1001, blk_1073741826_1002, blk_1073741827_1003]
[junit4] 2> 5115295 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 5115319 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema [halfcollection_shard1_replica1] Schema name=minimal
[junit4] 2> 5115327 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 5115328 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard1_replica1' using configuration from collection halfcollectionblocker
[junit4] 2> 5115329 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:45374/data
[junit4] 2> 5115329 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 5115329 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SolrCore [[halfcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-001/node3/halfcollection_shard1_replica1], dataDir=[hdfs://localhost:45374/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-004/]
[junit4] 2> 5115329 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-004/snapshot_metadata
[junit4] 2> 5115337 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5115338 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5115338 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5115349 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5115350 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-004
[junit4] 2> 5115370 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:45374/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001/tempDir-004/index
[junit4] 2> 5115377 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 5115377 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 5115377 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 5115389 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 5115416 INFO (IPC Server handler 1 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43585 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993:NORMAL:127.0.0.1:40559|RBW], ReplicaUC[[DISK]DS-2de90e67-b58c-488a-88cf-519b872ff0cb:NORMAL:127.0.0.1:43585|RBW]]} size 0
[junit4] 2> 5115417 INFO (IPC Server handler 4 on 45374) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40559 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1016e7ff-ca5b-4bc5-a254-e62816ae9993:NORMAL:127.0.0.1:40559|RBW], ReplicaUC[[DISK]DS-2de90e67-b58c-488a-88cf-519b872ff0cb:NORMAL:127.0.0.1:43585|RBW]]} size 0
[junit4] 2> 5115435 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 5115435 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5115436 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 5115445 INFO (qtp1409536826-85446) [n:127.0.0.1:46043_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.CommitTracke
[...truncated too long message...]
ans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 5265249 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59209 59209
[junit4] 2> 5265249 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F271390D7968BA19]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F271390D7968BA19-001
[junit4] 2> Nov 12, 2016 9:24:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=142, maxMBSortInHeap=6.165065154352495, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=cs, timezone=US/East-Indiana
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=311192608,total=516423680
[junit4] 2> NOTE: All tests run in this JVM: [BlockCacheTest, LeaderElectionContextKeyTest, CdcrVersionReplicationTest, TestBlobHandler, PKIAuthenticationIntegrationTest, TestQuerySenderNoQuery, TestChildDocTransformer, TestMacros, PeerSyncTest, ReplicationFactorTest, CreateCollectionCleanupTest, AnalysisErrorHandlingTest, DateRangeFieldTest, TestEmbeddedSolrServerConstructors, TestBadConfig, TestSerializedLuceneMatchVersion, TestComponentsName, ActionThrottleTest, TestFileDictionaryLookup, TestRandomRequestDistribution, BasicZkTest, DistributedQueryElevationComponentTest, TestReplicationHandler, SynonymTokenizerTest, CacheHeaderTest, NotRequiredUniqueKeyTest, SpatialRPTFieldTypeTest, SyncSliceTest, DistributedSpellCheckComponentTest, TestSolrCloudWithKerberosAlt, DisMaxRequestHandlerTest, HdfsRecoveryZkTest, ResourceLoaderTest, TestComplexPhraseQParserPlugin, SolrRequestParserTest, AnalyticsMergeStrategyTest, UniqFieldsUpdateProcessorFactoryTest, CursorPagingTest, TestPerFieldSimilarity, LeaderInitiatedRecoveryOnCommitTest, MergeStrategyTest, TestExactStatsCache, TestFieldCacheSort, TestFunctionQuery, BasicDistributedZkTest, TestAuthenticationFramework, TestOrdValues, SolrCLIZkUtilsTest, AtomicUpdatesTest, CloudExitableDirectoryReaderTest, HdfsThreadLeakTest, TestBinaryResponseWriter, TestJsonRequest, LeaderElectionIntegrationTest, TestFilteredDocIdSet, TestLockTree, TestIBSimilarityFactory, HdfsDirectoryTest, TestNoOpRegenerator, AnalysisAfterCoreReloadTest, DistributedFacetPivotWhiteBoxTest, TestJoin, TestJsonFacetRefinement, PathHierarchyTokenizerFactoryTest, DefaultValueUpdateProcessorTest, TestRequestForwarding, TestRandomCollapseQParserPlugin, TestClassicSimilarityFactory, SegmentsInfoRequestHandlerTest, SparseHLLTest, BadCopyFieldTest, TestMiniSolrCloudClusterSSL, PolyFieldTest, TriLevelCompositeIdRoutingTest, ExternalFileFieldSortTest, TestSchemaResource, TestSimpleQParserPlugin, TestSQLHandler, CheckHdfsIndexTest, TestRealTimeGet, TestJmxIntegration, TestCloudPivotFacet, TestCorePropertiesReload, DistribDocExpirationUpdateProcessorTest, TestFastLRUCache, TestRangeQuery, TestDeleteCollectionOnDownNodes, TestRandomDVFaceting, NumericFieldsTest, TestDynamicFieldResource, ShufflingReplicaListTransformerTest, TestImplicitCoreProperties, TestClusterProperties, TestFaceting, OutputWriterTest, HighlighterConfigTest, TestTrieFacet, TestManagedSchema, TestSolrCloudSnapshots, SharedFSAutoReplicaFailoverTest, TolerantUpdateProcessorTest, PeerSyncWithIndexFingerprintCachingTest, TestHdfsUpdateLog, DateFieldTest, TestOmitPositions, StandardRequestHandlerTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, OverseerTest, RecoveryZkTest, TestRandomFaceting, LeaderElectionTest, TestZkChroot, TestDistributedSearch, TestGroupingSearch, ShowFileRequestHandlerTest, CurrencyFieldXmlFileTest, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, DocValuesTest, DebugComponentTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, TermsComponentTest, RequestHandlersTest, TermVectorComponentTest, TestIndexingPerformance, RequiredFieldsTest, FastVectorHighlighterTest, RegexBoostProcessorTest, TestCollationField, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, TestPartialUpdateDeduplication, PingRequestHandlerTest, TestLFUCache, CdcrReplicationHandlerTest, CollectionReloadTest, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, ConnectionManagerTest, CustomCollectionTest, DeleteInactiveReplicaTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DeleteShardTest, DeleteStatusTest, DistribCursorPagingTest, DistributedQueueTest, DistributedVersionInfoTest, LeaderFailoverAfterPartitionTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnShardRestartTest, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestCryptoKeys, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestLocalFSCloudBackupRestore, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestShortCircuitedRequests, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [512/651 (1!)] on J0 in 173.61s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 61066 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 196 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/196/
4 tests failed.
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop
Error Message:
Timeout while trying to assert number of documents @ target_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ target_collection
at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:3B55C4154A256656]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: expected:<10> but was:<6>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:30C2255CF0018027]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:95E855DCA9D957D3]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={ "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-d554ffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"d5550000-2aa9ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"2aaa0000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:F0E36E3258CEC24A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11876 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/init-core-data-001
[junit4] 2> 1187228 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1187229 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001
[junit4] 2> 1187229 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1187230 INFO (Thread-1588) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1187230 INFO (Thread-1588) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1187330 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:51213
[junit4] 2> 1187338 INFO (jetty-launcher-711-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1187339 INFO (jetty-launcher-711-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1187338 INFO (jetty-launcher-711-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1187343 INFO (jetty-launcher-711-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1187345 INFO (jetty-launcher-711-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c2461d{/solr,null,AVAILABLE}
[junit4] 2> 1187351 INFO (jetty-launcher-711-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9f6cf6c{/solr,null,AVAILABLE}
[junit4] 2> 1187355 INFO (jetty-launcher-711-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@f8160a5{HTTP/1.1,[http/1.1]}{127.0.0.1:45888}
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.e.j.s.Server Started @1191337ms
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45888}
[junit4] 2> 1187356 ERROR (jetty-launcher-711-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1187356 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-11T19:45:55.207Z
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@4947e949{HTTP/1.1,[http/1.1]}{127.0.0.1:36630}
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.e.j.s.Server Started @1191338ms
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36630}
[junit4] 2> 1187357 ERROR (jetty-launcher-711-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1187357 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-11T19:45:55.208Z
[junit4] 2> 1187362 INFO (jetty-launcher-711-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7082d83e{/solr,null,AVAILABLE}
[junit4] 2> 1187362 INFO (jetty-launcher-711-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@72458de8{HTTP/1.1,[http/1.1]}{127.0.0.1:45902}
[junit4] 2> 1187362 INFO (jetty-launcher-711-thread-3) [ ] o.e.j.s.Server Started @1191344ms
[junit4] 2> 1187362 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45902}
[junit4] 2> 1187363 ERROR (jetty-launcher-711-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1187363 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1187363 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1187363 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1187363 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-11T19:45:55.214Z
[junit4] 2> 1187368 INFO (jetty-launcher-711-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@642cbee5{/solr,null,AVAILABLE}
[junit4] 2> 1187368 INFO (jetty-launcher-711-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@7e1b3b73{HTTP/1.1,[http/1.1]}{127.0.0.1:52143}
[junit4] 2> 1187368 INFO (jetty-launcher-711-thread-2) [ ] o.e.j.s.Server Started @1191350ms
[junit4] 2> 1187368 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52143}
[junit4] 2> 1187368 ERROR (jetty-launcher-711-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1187368 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1187369 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1187369 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1187369 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-11T19:45:55.220Z
[junit4] 2> 1187371 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1187378 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1187385 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1187386 INFO (jetty-launcher-711-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
[junit4] 2> 1187387 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1187394 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1187398 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1187398 INFO (jetty-launcher-711-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
[junit4] 2> 1187401 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1187401 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1187402 INFO (jetty-launcher-711-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
[junit4] 2> 1187403 INFO (jetty-launcher-711-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51213/solr
[junit4] 2> 1187450 INFO (jetty-launcher-711-thread-2) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52143_solr
[junit4] 2> 1187454 INFO (jetty-launcher-711-thread-3) [n:127.0.0.1:45902_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45902_solr
[junit4] 2> 1187455 INFO (jetty-launcher-711-thread-4) [n:127.0.0.1:36630_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36630_solr
[junit4] 2> 1187455 INFO (jetty-launcher-711-thread-1) [n:127.0.0.1:45888_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45888_solr
[junit4] 2> 1187455 INFO (jetty-launcher-711-thread-2) [n:127.0.0.1:52143_solr ] o.a.s.c.Overseer Overseer (id=96920768026312715-127.0.0.1:52143_solr-n_0000000000) starting
[junit4] 2> 1187471 INFO (zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) [n:127.0.0.1:45902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1187471 INFO (jetty-launcher-711-thread-4) [n:127.0.0.1:36630_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/.
[junit4] 2> 1187471 INFO (zkCallback-732-thread-1-processing-n:127.0.0.1:36630_solr) [n:127.0.0.1:36630_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1187472 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1187472 INFO (zkCallback-730-thread-1-processing-n:127.0.0.1:45888_solr) [n:127.0.0.1:45888_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1187481 INFO (jetty-launcher-711-thread-2) [n:127.0.0.1:52143_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52143_solr
[junit4] 2> 1187483 INFO (zkCallback-730-thread-1-processing-n:127.0.0.1:45888_solr) [n:127.0.0.1:45888_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1187483 INFO (zkCallback-732-thread-1-processing-n:127.0.0.1:36630_solr) [n:127.0.0.1:36630_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1187483 INFO (zkCallback-731-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1187489 INFO (jetty-launcher-711-thread-1) [n:127.0.0.1:45888_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node1/.
[junit4] 2> 1187492 INFO (jetty-launcher-711-thread-2) [n:127.0.0.1:52143_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node2/.
[junit4] 2> 1187498 INFO (zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) [n:127.0.0.1:45902_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1187527 INFO (jetty-launcher-711-thread-3) [n:127.0.0.1:45902_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node3/.
[junit4] 2> 1187534 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1188107 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1188780 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1188966 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 1188992 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1189064 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1189116 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_49480_hdfs____.a2omvj/webapp
[junit4] 2> 1189869 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49480
[junit4] 2> 1190777 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1190787 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1190802 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42718_datanode____.igor7d/webapp
[junit4] 2> 1191232 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42718
[junit4] 2> 1191702 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1191705 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1191723 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45598_datanode____.ihfcfa/webapp
[junit4] 2> 1192073 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45598
[junit4] 2> 1193206 INFO (IPC Server handler 5 on 37446) [ ] BlockStateChange BLOCK* processReport: from storage DS-f08d4d89-0367-4ff9-83f2-140475a4c65c node DatanodeRegistration(127.0.0.1:36702, datanodeUuid=1bf096c0-60fd-4e67-9f6f-f0ecf3732489, infoPort=54272, infoSecurePort=0, ipcPort=58312, storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 1193206 INFO (IPC Server handler 6 on 37446) [ ] BlockStateChange BLOCK* processReport: from storage DS-b4025d97-671c-49b1-a857-af62f730cb9d node DatanodeRegistration(127.0.0.1:45133, datanodeUuid=9dea7d69-f9e8-49d4-8582-792b2d73bf77, infoPort=37922, infoSecurePort=0, ipcPort=34614, storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 1193207 INFO (IPC Server handler 5 on 37446) [ ] BlockStateChange BLOCK* processReport: from storage DS-b7b2adcd-7dc2-4d4f-ac0a-e2e6c80efbc7 node DatanodeRegistration(127.0.0.1:36702, datanodeUuid=1bf096c0-60fd-4e67-9f6f-f0ecf3732489, infoPort=54272, infoSecurePort=0, ipcPort=58312, storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1193210 INFO (IPC Server handler 6 on 37446) [ ] BlockStateChange BLOCK* processReport: from storage DS-cfc90d9b-1fbd-46f8-ba06-ee243be158b7 node DatanodeRegistration(127.0.0.1:45133, datanodeUuid=9dea7d69-f9e8-49d4-8582-792b2d73bf77, infoPort=37922, infoSecurePort=0, ipcPort=34614, storageInfo=lv=-56;cid=testClusterID;nsid=919641284;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1193594 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBadActionNames
[junit4] 2> 1193597 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1193604 ERROR (qtp1903229075-5581) [n:127.0.0.1:36630_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Unknown action: BADACTION
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:203)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1193605 INFO (qtp1903229075-5581) [n:127.0.0.1:36630_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=badactioncollection&action=BADACTION&numShards=2&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 1193605 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testBadActionNames-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadActionNames
[junit4] 2> 1193672 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSpecificConfigsets
[junit4] 2> 1193674 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1193688 INFO (qtp1903229075-5583) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf2&name=withconfigset2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1193718 INFO (OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.CreateCollectionCmd Create collection withconfigset2
[junit4] 2> 1193836 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2
[junit4] 2> 1193942 INFO (zkCallback-729-thread-1-processing-n:127.0.0.1:45902_solr) [n:127.0.0.1:45902_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/withconfigset2/state.json] for collection [withconfigset2] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1194859 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 x:withconfigset2_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1194867 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 x:withconfigset2_shard1_replica1] o.a.s.s.IndexSchema [withconfigset2_shard1_replica1] Schema name=minimal
[junit4] 2> 1194870 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 x:withconfigset2_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1194872 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 x:withconfigset2_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'withconfigset2_shard1_replica1' using configuration from collection withconfigset2
[junit4] 2> 1194886 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
[junit4] 2> 1194886 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1194886 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.SolrCore [[withconfigset2_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node3/withconfigset2_shard1_replica1], dataDir=[hdfs://localhost:37446/data/withconfigset2/core_node1/data/]
[junit4] 2> 1194886 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8538112
[junit4] 2> 1194887 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.SolrCore [withconfigset2_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@5384e9c9
[junit4] 2> 1194887 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.SolrCore Error while closing
[junit4] 2> java.lang.NullPointerException
[junit4] 2> at org.apache.solr.core.SolrCore.close(SolrCore.java:1348)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:892)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1194888 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.c.CoreContainer Error creating core [withconfigset2_shard1_replica1]: Unrecognized lockType: native
[junit4] 2> org.apache.solr.common.SolrException: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:900)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.createLockFactory(HdfsDirectoryFactory.java:178)
[junit4] 2> at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
[junit4] 2> at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
[junit4] 2> ... 35 more
[junit4] 2> 1194890 ERROR (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error CREATEing SolrCore 'withconfigset2_shard1_replica1': Unable to create core [withconfigset2_shard1_replica1] Caused by: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:813)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unable to create core [withconfigset2_shard1_replica1]
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:855)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:779)
[junit4] 2> ... 32 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:900)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:775)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:842)
[junit4] 2> ... 33 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.createLockFactory(HdfsDirectoryFactory.java:178)
[junit4] 2> at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
[junit4] 2> at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:434)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:841)
[junit4] 2> ... 35 more
[junit4] 2>
[junit4] 2> 1194890 INFO (qtp64220878-5594) [n:127.0.0.1:45902_solr c:withconfigset2 s:shard1 r:core_node1 x:withconfigset2_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf2&newCollection=true&name=withconfigset2_shard1_replica1&action=CREATE&numShards=1&collection=withconfigset2&shard=shard1&wt=javabin&version=2} status=400 QTime=1056
[junit4] 2> 1194891 ERROR (OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:45902/solr
[junit4] 2> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45902/solr: Error CREATEing SolrCore 'withconfigset2_shard1_replica1': Unable to create core [withconfigset2_shard1_replica1] Caused by: Unrecognized lockType: native
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:196)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1194895 ERROR (OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [withconfigset2].
[junit4] 2> 1194896 INFO (OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1194898 INFO (qtp64220878-5596) [n:127.0.0.1:45902_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=withconfigset2_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1195601 INFO (OverseerThreadFactory-3109-thread-1-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [withconfigset2]
[junit4] 2> 1195605 INFO (qtp1903229075-5583) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1195700 INFO (OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1225610 ERROR (qtp1903229075-5583) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE with timeout=30
[junit4] 2> 1225611 INFO (qtp1903229075-5583) [n:127.0.0.1:36630_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf2&name=withconfigset2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=31922
[junit4] 2> 1225612 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSpecificConfigsets
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.method=testSpecificConfigsets -Dtests.seed=B8961A865EFDEDDF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=ms-MY -Dtests.timezone=Asia/Kabul -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 32.0s J0 | HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
[junit4] > at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:95E855DCA9D957D3]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
[junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1225692 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
[junit4] 2> 1225694 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1225697 INFO (qtp1903229075-5584) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1225700 INFO (OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.CreateCollectionCmd Create collection collection
[junit4] 2> 1225700 WARN (OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 1225701 ERROR (OverseerThreadFactory-3109-thread-2-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1225704 INFO (qtp1903229075-5584) [n:127.0.0.1:36630_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=6
[junit4] 2> 1225704 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
[junit4] 2> 1225788 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMissingNumShards
[junit4] 2> 1225789 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1225793 INFO (qtp1903229075-5586) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1225798 INFO (OverseerThreadFactory-3109-thread-3-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.CreateCollectionCmd Create collection acollection
[junit4] 2> 1225799 ERROR (OverseerThreadFactory-3109-thread-3-processing-n:127.0.0.1:52143_solr) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: acollection operation: create failed:org.apache.solr.common.SolrException: numShards is a required param (when using CompositeId router).
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:105)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1225802 INFO (OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1225808 INFO (qtp1903229075-5586) [n:127.0.0.1:36630_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=acollection&action=CREATE&wt=javabin&version=2} status=400 QTime=14
[junit4] 2> 1225808 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testMissingNumShards-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMissingNumShards
[junit4] 2> 1225900 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testNoCollectionSpecified-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNoCollectionSpecified
[junit4] 2> 1225901 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testNoCollectionSpecified-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1225905 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection&action=CREATE&numShards=1&collection=&wt=javabin&version=2
[junit4] 2> 1225908 INFO (OverseerStateUpdate-96920768026312715-127.0.0.1:52143_solr-n_0000000000) [n:127.0.0.1:52143_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1226919 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection x:corewithnocollection] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1226947 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection x:corewithnocollection] o.a.s.s.IndexSchema [corewithnocollection] Schema name=minimal
[junit4] 2> 1226950 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection x:corewithnocollection] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1226951 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection x:corewithnocollection] o.a.s.c.CoreContainer Creating SolrCore 'corewithnocollection' using configuration from collection corewithnocollection
[junit4] 2> 1226957 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
[junit4] 2> 1226958 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1226958 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.SolrCore [[corewithnocollection] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/corewithnocollection], dataDir=[hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/]
[junit4] 2> 1226959 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/snapshot_metadata
[junit4] 2> 1226978 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1226979 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1226979 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 1227805 INFO (OverseerCollectionConfigSetProcessor-96920768026312715-127.0.0.1:52143_solr-n_0000000000) [n:127.0.0.1:52143_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1228647 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1228656 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003
[junit4] 2> 1228694 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/index
[junit4] 2> 1228702 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1228702 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1228709 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1229027 INFO (IPC Server handler 4 on 37446) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36702 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW], ReplicaUC[[DISK]DS-b7b2adcd-7dc2-4d4f-ac0a-e2e6c80efbc7:NORMAL:127.0.0.1:36702|RBW]]} size 71
[junit4] 2> 1229037 INFO (IPC Server handler 7 on 37446) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45133 is added to blk_1073741825_1001 size 71
[junit4] 2> 1229480 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1229481 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1229481 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1229498 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1229498 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1229595 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.s.SolrIndexSearcher Opening [Searcher@34ab42d[corewithnocollection] main]
[junit4] 2> 1229596 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1229596 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1229599 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1229599 INFO (searcherExecutor-3117-thread-1-processing-n:127.0.0.1:36630_solr x:corewithnocollection s:shard1 c:corewithnocollection r:core_node1) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.SolrCore [corewithnocollection] Registered new searcher Searcher@34ab42d[corewithnocollection] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1229599 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550732332839731200
[junit4] 2> 1229612 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1229612 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1229612 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36630/solr/corewithnocollection/
[junit4] 2> 1229612 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1229612 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.SyncStrategy http://127.0.0.1:36630/solr/corewithnocollection/ has no replicas
[junit4] 2> 1229625 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36630/solr/corewithnocollection/ shard1
[junit4] 2> 1229778 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1229780 INFO (qtp1903229075-5585) [n:127.0.0.1:36630_solr c:corewithnocollection s:shard1 r:core_node1 x:corewithnocollection] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection&action=CREATE&numShards=1&collection=&wt=javabin&version=2} status=0 QTime=3875
[junit4] 2> 1229783 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003&collection.configName=conf&name=corewithnocollection2&action=CREATE&numShards=1&wt=javabin&version=2
[junit4] 2> 1229787 INFO (OverseerStateUpdate-96920768026312715-127.0.0.1:52143_solr-n_0000000000) [n:127.0.0.1:52143_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1230804 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 x:corewithnocollection2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1230813 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 x:corewithnocollection2] o.a.s.s.IndexSchema [corewithnocollection2] Schema name=minimal
[junit4] 2> 1230816 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 x:corewithnocollection2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1230817 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 x:corewithnocollection2] o.a.s.c.CoreContainer Creating SolrCore 'corewithnocollection2' using configuration from collection corewithnocollection2
[junit4] 2> 1230818 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37446/data
[junit4] 2> 1230818 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1230818 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.SolrCore [[corewithnocollection2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-001/node4/corewithnocollection2], dataDir=[hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/]
[junit4] 2> 1230818 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/snapshot_metadata
[junit4] 2> 1230828 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1230828 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1230854 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1230856 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003
[junit4] 2> 1230923 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37446/data/corewithnocollection2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_B8961A865EFDEDDF-001/tempDir-003/index
[junit4] 2> 1230931 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1230931 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1230937 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1231014 INFO (IPC Server handler 5 on 37446) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45133 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f08d4d89-0367-4ff9-83f2-140475a4c65c:NORMAL:127.0.0.1:36702|RBW], ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW]]} size 0
[junit4] 2> 1231015 INFO (IPC Server handler 5 on 37446) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36702 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f08d4d89-0367-4ff9-83f2-140475a4c65c:NORMAL:127.0.0.1:36702|RBW], ReplicaUC[[DISK]DS-b4025d97-671c-49b1-a857-af62f730cb9d:NORMAL:127.0.0.1:45133|RBW]]} size 0
[junit4] 2> 1231059 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1231059 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1231059 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithnocollection2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1231084 INFO (qtp1903229075-5572) [n:127.0.0.1:36630_solr c:corewithnocollection2 s:shard1 r:core_node1 x:corewithn
[...truncated too long message...]
[B8961A865EFDEDDF]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@710f30a4{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 4528214 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@56ee4cf9{/,null,UNAVAILABLE}
[junit4] 2> 4528215 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReplicationStartStop
[junit4] 2> 4528216 INFO (TEST-CdcrReplicationDistributedZkTest.testReplicationStartStop-seed#[B8961A865EFDEDDF]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52193 52193
[junit4] 2> 4528322 INFO (Thread-112192) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52193 52193
[junit4] 2> 4528323 WARN (Thread-112192) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 26 /solr/aliases.json
[junit4] 2> 16 /solr/configs/conf1
[junit4] 2> 8 /solr/collections/tmp_collection/state.json
[junit4] 2> 8 /solr/security.json
[junit4] 2> 3 /solr/collections/target_collection/leaders/shard2/leader
[junit4] 2> 3 /solr/collections/target_collection/leaders/shard1/leader
[junit4] 2> 2 /solr/collections/source_collection/leaders/shard1/leader
[junit4] 2> 2 /solr/collections/source_collection/leaders/shard2/leader
[junit4] 2> 2 /solr/collections/tmp_collection/leaders/shard2/leader
[junit4] 2> 2 /solr/collections/tmp_collection/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 26 /solr/clusterprops.json
[junit4] 2> 26 /solr/clusterstate.json
[junit4] 2> 8 /solr/collections/tmp_collection/cdcr/state/process
[junit4] 2> 8 /solr/collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 6 /solr/collections/target_collection/cdcr/state/buffer
[junit4] 2> 6 /solr/collections/target_collection/cdcr/state/process
[junit4] 2> 5 /solr/collections/source_collection/cdcr/state/process
[junit4] 2> 4 /solr/collections/source_collection/state.json
[junit4] 2> 4 /solr/collections/source_collection/cdcr/state/buffer
[junit4] 2> 4 /solr/collections/target_collection/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 175 /solr/overseer/collection-queue-work
[junit4] 2> 63 /solr/overseer/queue
[junit4] 2> 26 /solr/live_nodes
[junit4] 2> 26 /solr/collections
[junit4] 2> 21 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CdcrReplicationDistributedZkTest -Dtests.method=testReplicationStartStop -Dtests.seed=B8961A865EFDEDDF -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=es-MX -Dtests.timezone=US/Hawaii -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 188s J0 | CdcrReplicationDistributedZkTest.testReplicationStartStop <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout while trying to assert number of documents @ target_collection
[junit4] > at __randomizedtesting.SeedInfo.seed([B8961A865EFDEDDF:3B55C4154A256656]:0)
[junit4] > at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
[junit4] > at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:173)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.AssertionError: expected:<10> but was:<6>
[junit4] > at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
[junit4] > ... 42 more
[junit4] 2> 4528325 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[B8961A865EFDEDDF]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_B8961A865EFDEDDF-001
[junit4] 2> Nov 11, 2016 8:41:36 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {_version_=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=Asserting)}, docValues:{}, maxPointsInLeafNode=632, maxMBSortInHeap=6.508491660832644, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=es-MX, timezone=US/Hawaii
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=220688288,total=522190848
[junit4] 2> NOTE: All tests run in this JVM: [CloudExitableDirectoryReaderTest, CacheHeaderTest, DistribDocExpirationUpdateProcessorTest, NumericFieldsTest, TestLazyCores, FullHLLTest, PeerSyncTest, MergeStrategyTest, TestInitQParser, TestSchemaResource, DistributedExpandComponentTest, TestSolrCoreProperties, TestBinaryResponseWriter, TriLevelCompositeIdRoutingTest, RequiredFieldsTest, UniqFieldsUpdateProcessorFactoryTest, QueryResultKeyTest, TestMacros, TestExclusionRuleCollectionAccess, TestPHPSerializedResponseWriter, TestReplicationHandler, TestRandomDVFaceting, FastVectorHighlighterTest, GraphQueryTest, SSLMigrationTest, DeleteLastCustomShardedReplicaTest, TestPerFieldSimilarity, TestConfigSetImmutable, TestSchemaManager, TestNRTOpen, TestReplicationHandlerBackup, PathHierarchyTokenizerFactoryTest, PolyFieldTest, UUIDFieldTest, AnalyticsMergeStrategyTest, ReturnFieldsTest, TestExactStatsCache, TestBlobHandler, MultiThreadedOCPTest, SuggestComponentTest, AsyncCallRequestStatusResponseTest, CollectionTooManyReplicasTest, TestRandomFaceting, TestPerFieldSimilarityClassic, LeaderElectionContextKeyTest, AnalysisAfterCoreReloadTest, TestSubQueryTransformerDistrib, HdfsCollectionsAPIDistributedZkTest, TestFaceting, HdfsBasicDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, FileBasedSpellCheckerTest, TestCrossCoreJoin, TestHdfsUpdateLog, TestRandomRequestDistribution, RuleEngineTest, HighlighterMaxOffsetTest, DisMaxRequestHandlerTest, TestRemoteStreaming, TestDocumentBuilder, TestSolrFieldCacheMBean, ActionThrottleTest, TestClusterProperties, RecoveryAfterSoftCommitTest, TestDocSet, DocumentAnalysisRequestHandlerTest, TestManagedSchema, TestFilteredDocIdSet, OverseerStatusTest, CursorPagingTest, TestJmxIntegration, TestOrdValues, TestRecoveryHdfs, DistributedVersionInfoTest, SolrCoreTest, TestShortCircuitedRequests, RegexBoostProcessorTest, BlockCacheTest, TestFieldCacheWithThreads, TestSolrConfigHandler, AtomicUpdatesTest, CurrencyFieldOpenExchangeTest, DistributedSuggestComponentTest, TestSolrDeletionPolicy2, TestOmitPositions, TestCloudPivotFacet, TestDistributedSearch, TestPivotHelperCode, CheckHdfsIndexTest, QueryElevationComponentTest, TestFoldingMultitermQuery, TestRuleBasedAuthorizationPlugin, TestComponentsName, TestRandomCollapseQParserPlugin, TestConfigSetsAPIExclusivity, AlternateDirectoryTest, TestQuerySenderNoQuery, ZkNodePropsTest, LeaderElectionIntegrationTest, TestBinaryResponseWriter, ClusterStateTest, HdfsRecoveryZkTest, DirectoryFactoryTest, TestRequestForwarding, TestMiniSolrCloudClusterSSL, SimpleFacetsTest, TestSolr4Spatial, SolrCmdDistributorTest, BadIndexSchemaTest, TestSort, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, DirectUpdateHandlerTest, TestBadConfig, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, SolrRequestParserTest, DocValuesTest, TestStressLucene, TestCSVLoader, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, QueryEqualityTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, DocValuesMultiTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, TestQueryTypes, PrimitiveFieldTypeTest, RequestHandlersTest, TestValueSourceCache, TermVectorComponentTest, TestIndexingPerformance, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, IndexSchemaTest, TestCollationField, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, SOLR749Test, TestSolrIndexConfig, CopyFieldTest, SolrIndexConfigTest, TestStressRecovery, PreAnalyzedFieldTest, TestSystemIdResolver, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestLRUCache, TestUtils, CircularListTest, TestRTGBase, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, BaseCdcrDistributedZkTest, CdcrReplicationDistributedZkTest]
[junit4] Completed [501/651 (3!)] on J0 in 1676.63s, 11 tests, 1 failure <<< FAILURES!
[...truncated 61103 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 195 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/195/
3 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes
Error Message:
Could not find collection : nodes_used_collection
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : nodes_used_collection
at __randomizedtesting.SeedInfo.seed([F6C1D2403434BA00:9281072C8FFA3211]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:193)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:216)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes(CollectionsAPIDistributedZkTest.java:397)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={ "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-d554ffff", "state":"active", "replicas":{}}, "shard2":{ "range":"d5550000-2aa9ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"2aaa0000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-d554ffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"d5550000-2aa9ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"2aaa0000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([F6C1D2403434BA00:BEB4A6F432079595]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([F6C1D2403434BA00:DBBF9D1AC310000C]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11384 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/init-core-data-001
[junit4] 2> 345438 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 345439 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001
[junit4] 2> 345439 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 345447 INFO (Thread-340) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 345447 INFO (Thread-340) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 345547 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:47793
[junit4] 2> 345623 INFO (jetty-launcher-223-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 345623 INFO (jetty-launcher-223-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 345623 INFO (jetty-launcher-223-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 345626 INFO (jetty-launcher-223-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 345633 INFO (jetty-launcher-223-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@300ac283{/solr,null,AVAILABLE}
[junit4] 2> 345640 INFO (jetty-launcher-223-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@663b3020{/solr,null,AVAILABLE}
[junit4] 2> 345643 INFO (jetty-launcher-223-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@13af7c5e{SSL,[ssl, http/1.1]}{127.0.0.1:48954}
[junit4] 2> 345644 INFO (jetty-launcher-223-thread-3) [ ] o.e.j.s.Server Started @350380ms
[junit4] 2> 345644 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48954}
[junit4] 2> 345644 ERROR (jetty-launcher-223-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 345659 INFO (jetty-launcher-223-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@143ffe37{/solr,null,AVAILABLE}
[junit4] 2> 345672 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 345672 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 345672 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 345672 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-10T16:33:56.830Z
[junit4] 2> 345681 INFO (jetty-launcher-223-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@1c7e9c8b{SSL,[ssl, http/1.1]}{127.0.0.1:48587}
[junit4] 2> 345681 INFO (jetty-launcher-223-thread-1) [ ] o.e.j.s.Server Started @350419ms
[junit4] 2> 345681 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48587}
[junit4] 2> 345682 ERROR (jetty-launcher-223-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@4b9d5386{SSL,[ssl, http/1.1]}{127.0.0.1:40953}
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.e.j.s.Server Started @350420ms
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40953}
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-10T16:33:56.840Z
[junit4] 2> 345682 ERROR (jetty-launcher-223-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 345682 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-10T16:33:56.840Z
[junit4] 2> 345692 INFO (jetty-launcher-223-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10cc5291{/solr,null,AVAILABLE}
[junit4] 2> 345708 INFO (jetty-launcher-223-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@211452e2{SSL,[ssl, http/1.1]}{127.0.0.1:60566}
[junit4] 2> 345708 INFO (jetty-launcher-223-thread-4) [ ] o.e.j.s.Server Started @350446ms
[junit4] 2> 345708 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60566}
[junit4] 2> 345715 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 345722 ERROR (jetty-launcher-223-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 345722 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 345722 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 345722 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 345722 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-10T16:33:56.880Z
[junit4] 2> 345724 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 345724 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 345747 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 345747 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 345748 INFO (jetty-launcher-223-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47793/solr
[junit4] 2> 345757 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 345763 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 345764 INFO (jetty-launcher-223-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47793/solr
[junit4] 2> 345772 INFO (jetty-launcher-223-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47793/solr
[junit4] 2> 345773 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 345776 INFO (jetty-launcher-223-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:47793/solr
[junit4] 2> 345838 INFO (jetty-launcher-223-thread-4) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60566_solr
[junit4] 2> 345851 INFO (jetty-launcher-223-thread-3) [n:127.0.0.1:48954_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48954_solr
[junit4] 2> 345852 INFO (jetty-launcher-223-thread-4) [n:127.0.0.1:60566_solr ] o.a.s.c.Overseer Overseer (id=96914350840610826-127.0.0.1:60566_solr-n_0000000000) starting
[junit4] 2> 345858 INFO (zkCallback-242-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 345858 INFO (zkCallback-240-thread-1-processing-n:127.0.0.1:48954_solr) [n:127.0.0.1:48954_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 345860 INFO (jetty-launcher-223-thread-1) [n:127.0.0.1:48587_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 345875 INFO (jetty-launcher-223-thread-2) [n:127.0.0.1:40953_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 345970 INFO (jetty-launcher-223-thread-4) [n:127.0.0.1:60566_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60566_solr
[junit4] 2> 345970 INFO (jetty-launcher-223-thread-3) [n:127.0.0.1:48954_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node3/.
[junit4] 2> 345973 INFO (jetty-launcher-223-thread-1) [n:127.0.0.1:48587_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48587_solr
[junit4] 2> 345977 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:40953_solr) [n:127.0.0.1:40953_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 345978 INFO (zkCallback-244-thread-1-processing-n:127.0.0.1:48587_solr) [n:127.0.0.1:48587_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 345979 INFO (zkCallback-242-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 345979 INFO (zkCallback-240-thread-1-processing-n:127.0.0.1:48954_solr) [n:127.0.0.1:48954_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 345993 INFO (jetty-launcher-223-thread-2) [n:127.0.0.1:40953_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40953_solr
[junit4] 2> 345994 INFO (jetty-launcher-223-thread-4) [n:127.0.0.1:60566_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node4/.
[junit4] 2> 345998 INFO (zkCallback-242-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 345998 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:40953_solr) [n:127.0.0.1:40953_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 345998 INFO (zkCallback-240-thread-1-processing-n:127.0.0.1:48954_solr) [n:127.0.0.1:48954_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 345998 INFO (zkCallback-244-thread-1-processing-n:127.0.0.1:48587_solr) [n:127.0.0.1:48587_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 346010 INFO (jetty-launcher-223-thread-1) [n:127.0.0.1:48587_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node1/.
[junit4] 2> 346016 INFO (jetty-launcher-223-thread-2) [n:127.0.0.1:40953_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node2/.
[junit4] 2> 346060 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 347077 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 348278 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 348704 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 348740 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 348842 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 348963 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_40328_hdfs____.6425qx/webapp
[junit4] 2> 349936 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40328
[junit4] 2> 351166 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 351173 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 351188 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52013_datanode____.4a7rpg/webapp
[junit4] 2> 351695 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52013
[junit4] 2> 352263 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 352265 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 352299 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_44939_datanode____.72620q/webapp
[junit4] 2> 352814 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44939
[junit4] 2> 354590 INFO (IPC Server handler 5 on 53630) [ ] BlockStateChange BLOCK* processReport: from storage DS-83749173-1659-4204-a1f8-b6154a34c35b node DatanodeRegistration(127.0.0.1:42118, datanodeUuid=10445a2a-eb7f-44dd-a193-854f45793606, infoPort=58519, infoSecurePort=0, ipcPort=59056, storageInfo=lv=-56;cid=testClusterID;nsid=804710585;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 354595 INFO (IPC Server handler 6 on 53630) [ ] BlockStateChange BLOCK* processReport: from storage DS-c6da2482-99f6-4508-8d53-ae53cbe10e92 node DatanodeRegistration(127.0.0.1:40732, datanodeUuid=6c5428c7-2126-40d8-82f5-5abb1bf54001, infoPort=46788, infoSecurePort=0, ipcPort=55323, storageInfo=lv=-56;cid=testClusterID;nsid=804710585;c=0), blocks: 0, hasStaleStorage: true, processing time: 4 msecs
[junit4] 2> 354595 INFO (IPC Server handler 6 on 53630) [ ] BlockStateChange BLOCK* processReport: from storage DS-10654d7c-36e4-4e34-9173-6487d189d1cc node DatanodeRegistration(127.0.0.1:40732, datanodeUuid=6c5428c7-2126-40d8-82f5-5abb1bf54001, infoPort=46788, infoSecurePort=0, ipcPort=55323, storageInfo=lv=-56;cid=testClusterID;nsid=804710585;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 354596 INFO (IPC Server handler 5 on 53630) [ ] BlockStateChange BLOCK* processReport: from storage DS-446fe27d-3df2-46a9-be9d-86c6dbb4bff1 node DatanodeRegistration(127.0.0.1:42118, datanodeUuid=10445a2a-eb7f-44dd-a193-854f45793606, infoPort=58519, infoSecurePort=0, ipcPort=59056, storageInfo=lv=-56;cid=testClusterID;nsid=804710585;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
[junit4] 2> 354752 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[F6C1D2403434BA00]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
[junit4] 2> 354754 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[F6C1D2403434BA00]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 354831 INFO (qtp516410542-1593) [n:127.0.0.1:60566_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 354837 INFO (OverseerThreadFactory-656-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.CreateCollectionCmd Create collection collection
[junit4] 2> 354838 WARN (OverseerThreadFactory-656-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 354841 ERROR (OverseerThreadFactory-656-thread-1-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 354866 INFO (qtp516410542-1593) [n:127.0.0.1:60566_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=35
[junit4] 2> 354869 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[F6C1D2403434BA00]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
[junit4] 2> 354965 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[F6C1D2403434BA00]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateShouldFailOnExistingCore
[junit4] 2> 354967 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[F6C1D2403434BA00]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 355043 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003&collection.configName=conf&name=halfcollection_shard1_replica1&action=CREATE&numShards=1&collection=halfcollectionblocker&wt=javabin&version=2
[junit4] 2> 355046 INFO (OverseerStateUpdate-96914350840610826-127.0.0.1:60566_solr-n_0000000000) [n:127.0.0.1:60566_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 355103 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 355117 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema [halfcollection_shard1_replica1] Schema name=minimal
[junit4] 2> 355121 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 355122 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker x:halfcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard1_replica1' using configuration from collection halfcollectionblocker
[junit4] 2> 355143 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53630/data
[junit4] 2> 355143 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 355144 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SolrCore [[halfcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node3/halfcollection_shard1_replica1], dataDir=[hdfs://localhost:53630/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003/]
[junit4] 2> 355145 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003/snapshot_metadata
[junit4] 2> 355172 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 355173 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 355173 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 356839 INFO (OverseerCollectionConfigSetProcessor-96914350840610826-127.0.0.1:60566_solr-n_0000000000) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 357238 WARN (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 362241 WARN (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 367537 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 367549 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003
[junit4] 2> 367663 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003/index
[junit4] 2> 367676 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 367676 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 367676 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 367690 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 369162 INFO (IPC Server handler 2 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40732 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-10654d7c-36e4-4e34-9173-6487d189d1cc:NORMAL:127.0.0.1:40732|RBW], ReplicaUC[[DISK]DS-446fe27d-3df2-46a9-be9d-86c6dbb4bff1:NORMAL:127.0.0.1:42118|RBW]]} size 71
[junit4] 2> 369164 INFO (IPC Server handler 3 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42118 is added to blk_1073741825_1001 size 71
[junit4] 2> 369594 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 369594 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 369594 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 369627 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 369627 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 369763 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1334f2a7[halfcollection_shard1_replica1] main]
[junit4] 2> 369764 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 369765 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 369768 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 369768 INFO (searcherExecutor-663-thread-1-processing-n:127.0.0.1:48954_solr x:halfcollection_shard1_replica1 s:shard1 c:halfcollectionblocker r:core_node1) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SolrCore [halfcollection_shard1_replica1] Registered new searcher Searcher@1334f2a7[halfcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 369768 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550629638951141376
[junit4] 2> 369787 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 369787 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 369787 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:48954/solr/halfcollection_shard1_replica1/
[junit4] 2> 369787 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 369787 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:48954/solr/halfcollection_shard1_replica1/ has no replicas
[junit4] 2> 369799 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:48954/solr/halfcollection_shard1_replica1/ shard1
[junit4] 2> 369955 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 369957 INFO (qtp1362840958-1574) [n:127.0.0.1:48954_solr c:halfcollectionblocker s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-003&collection.configName=conf&name=halfcollection_shard1_replica1&action=CREATE&numShards=1&collection=halfcollectionblocker&wt=javabin&version=2} status=0 QTime=14914
[junit4] 2> 370031 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.h.a.CoreAdminOperation core create command dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004&collection.configName=conf&name=halfcollection_shard1_replica1&action=CREATE&numShards=1&collection=halfcollectionblocker2&wt=javabin&version=2
[junit4] 2> 370034 INFO (OverseerStateUpdate-96914350840610826-127.0.0.1:60566_solr-n_0000000000) [n:127.0.0.1:60566_solr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 371054 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 x:halfcollection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 371063 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema [halfcollection_shard1_replica1] Schema name=minimal
[junit4] 2> 371067 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 x:halfcollection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 371068 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 x:halfcollection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard1_replica1' using configuration from collection halfcollectionblocker2
[junit4] 2> 371069 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53630/data
[junit4] 2> 371070 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 371070 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SolrCore [[halfcollection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node2/halfcollection_shard1_replica1], dataDir=[hdfs://localhost:53630/data/halfcollectionblocker2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004/]
[junit4] 2> 371070 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004/snapshot_metadata
[junit4] 2> 371086 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 371086 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 371086 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 371104 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 371107 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004
[junit4] 2> 371152 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollectionblocker2/core_node1//x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004/index
[junit4] 2> 371163 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 371163 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 371163 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 371184 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 371294 INFO (IPC Server handler 0 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42118 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-10654d7c-36e4-4e34-9173-6487d189d1cc:NORMAL:127.0.0.1:40732|RBW], ReplicaUC[[DISK]DS-446fe27d-3df2-46a9-be9d-86c6dbb4bff1:NORMAL:127.0.0.1:42118|RBW]]} size 0
[junit4] 2> 371302 INFO (IPC Server handler 8 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40732 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-10654d7c-36e4-4e34-9173-6487d189d1cc:NORMAL:127.0.0.1:40732|RBW], ReplicaUC[[DISK]DS-446fe27d-3df2-46a9-be9d-86c6dbb4bff1:NORMAL:127.0.0.1:42118|RBW]]} size 0
[junit4] 2> 371327 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 371327 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 371327 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 371362 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 371363 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 371407 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@37d5b409[halfcollection_shard1_replica1] main]
[junit4] 2> 371409 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 371409 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 371409 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 371410 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550629640672903168
[junit4] 2> 371410 INFO (searcherExecutor-668-thread-1-processing-n:127.0.0.1:40953_solr x:halfcollection_shard1_replica1 s:shard1 c:halfcollectionblocker2 r:core_node1) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SolrCore [halfcollection_shard1_replica1] Registered new searcher Searcher@37d5b409[halfcollection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 371426 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 371427 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 371427 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40953/solr/halfcollection_shard1_replica1/
[junit4] 2> 371427 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 371427 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:40953/solr/halfcollection_shard1_replica1/ has no replicas
[junit4] 2> 371444 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40953/solr/halfcollection_shard1_replica1/ shard1
[junit4] 2> 371599 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 371603 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr c:halfcollectionblocker2 s:shard1 r:core_node1 x:halfcollection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={dataDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004&collection.configName=conf&name=halfcollection_shard1_replica1&action=CREATE&numShards=1&collection=halfcollectionblocker2&wt=javabin&version=2} status=0 QTime=1572
[junit4] 2> 371608 INFO (qtp516410542-1588) [n:127.0.0.1:60566_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=halfcollection&action=CREATE&numShards=2&createNodeSet=127.0.0.1:48954_solr,127.0.0.1:40953_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 371616 INFO (OverseerThreadFactory-656-thread-2-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.CreateCollectionCmd Create collection halfcollection
[junit4] 2> 371847 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfcollection&shard=shard2&wt=javabin&version=2
[junit4] 2> 371869 INFO (qtp1362840958-1573) [n:127.0.0.1:48954_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfcollection&shard=shard1&wt=javabin&version=2
[junit4] 2> 371870 WARN (qtp1362840958-1573) [n:127.0.0.1:48954_solr ] o.a.s.c.CoreContainer Creating a core with existing name is not allowed
[junit4] 2> 371870 ERROR (qtp1362840958-1573) [n:127.0.0.1:48954_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Core with name 'halfcollection_shard1_replica1' already exists.
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:764)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 371870 INFO (qtp1362840958-1573) [n:127.0.0.1:48954_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfcollection_shard1_replica1&action=CREATE&numShards=2&collection=halfcollection&shard=shard1&wt=javabin&version=2} status=500 QTime=1
[junit4] 2> 371871 ERROR (qtp1362840958-1573) [n:127.0.0.1:48954_solr ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Core with name 'halfcollection_shard1_replica1' already exists.
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:764)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:88)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:377)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:365)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:156)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:660)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:441)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:110)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:518)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 371873 ERROR (OverseerThreadFactory-656-thread-2-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: https://127.0.0.1:48954/solr
[junit4] 2> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:48954/solr: Core with name 'halfcollection_shard1_replica1' already exists.
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:593)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:196)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 371954 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:40953_solr) [n:127.0.0.1:40953_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollection/state.json] for collection [halfcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 372867 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 x:halfcollection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 372877 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 x:halfcollection_shard2_replica1] o.a.s.s.IndexSchema [halfcollection_shard2_replica1] Schema name=minimal
[junit4] 2> 372880 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 x:halfcollection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 372881 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 x:halfcollection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'halfcollection_shard2_replica1' using configuration from collection halfcollection
[junit4] 2> 372882 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:53630/data
[junit4] 2> 372882 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 372882 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.SolrCore [[halfcollection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-001/node2/halfcollection_shard2_replica1], dataDir=[hdfs://localhost:53630/data/halfcollection/core_node1/data/]
[junit4] 2> 372882 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollection/core_node1/data/snapshot_metadata
[junit4] 2> 372890 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 372890 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 372890 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 372902 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 372904 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollection/core_node1/data
[junit4] 2> 372934 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:53630/data/halfcollection/core_node1/data/index
[junit4] 2> 372942 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 372942 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 372942 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 372955 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 373009 INFO (IPC Server handler 8 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42118 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c6da2482-99f6-4508-8d53-ae53cbe10e92:NORMAL:127.0.0.1:40732|RBW], ReplicaUC[[DISK]DS-83749173-1659-4204-a1f8-b6154a34c35b:NORMAL:127.0.0.1:42118|FINALIZED]]} size 0
[junit4] 2> 373010 INFO (IPC Server handler 9 on 53630) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40732 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c6da2482-99f6-4508-8d53-ae53cbe10e92:NORMAL:127.0.0.1:40732|RBW], ReplicaUC[[DISK]DS-83749173-1659-4204-a1f8-b6154a34c35b:NORMAL:127.0.0.1:42118|FINALIZED]]} size 0
[junit4] 2> 373051 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 373051 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 373051 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 373074 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 373074 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 373110 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@51ca7b8a[halfcollection_shard2_replica1] main]
[junit4] 2> 373113 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 373113 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 373114 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 373114 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550629642459676672
[junit4] 2> 373118 INFO (searcherExecutor-673-thread-1-processing-n:127.0.0.1:40953_solr x:halfcollection_shard2_replica1 s:shard2 c:halfcollection r:core_node1) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.SolrCore [halfcollection_shard2_replica1] Registered new searcher Searcher@51ca7b8a[halfcollection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 373129 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 373129 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 373130 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40953/solr/halfcollection_shard2_replica1/
[junit4] 2> 373130 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 373130 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:40953/solr/halfcollection_shard2_replica1/ has no replicas
[junit4] 2> 373136 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40953/solr/halfcollection_shard2_replica1/ shard2
[junit4] 2> 373241 INFO (zkCallback-243-thread-1-processing-n:127.0.0.1:40953_solr) [n:127.0.0.1:40953_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/halfcollection/state.json] for collection [halfcollection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 373289 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 373291 INFO (qtp89624268-1579) [n:127.0.0.1:40953_solr c:halfcollection s:shard2 r:core_node1 x:halfcollection_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=halfcollection_shard2_replica1&action=CREATE&numShards=2&collection=halfcollection&shard=shard2&wt=javabin&version=2} status=0 QTime=1444
[junit4] 2> 373292 ERROR (OverseerThreadFactory-656-thread-2-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [halfcollection].
[junit4] 2> 373292 INFO (OverseerThreadFactory-656-thread-2-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 373296 INFO (qtp89624268-1558) [n:127.0.0.1:40953_solr ] o.a.s.c.SolrCore [halfcollection_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@5cacd77f
[junit4] 2> 373310 INFO (qtp89624268-1558) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollection/core_node1/data/index
[junit4] 2> 373313 INFO (qtp89624268-1558) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollection/core_node1/data/snapshot_metadata
[junit4] 2> 373313 INFO (qtp89624268-1558) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollection/core_node1/data
[junit4] 2> 373326 INFO (IPC Server handler 7 on 53630) [ ] BlockStateChange BLOCK* addToInvalidates: blk_1073741827_1003 127.0.0.1:40732 127.0.0.1:42118
[junit4] 2> 373345 INFO (qtp89624268-1558) [n:127.0.0.1:40953_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfcollection_shard2_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=50
[junit4] 2> 374049 INFO (OverseerThreadFactory-656-thread-2-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.CreateCollectionCmd Cleaned up artifacts for failed create collection for [halfcollection]
[junit4] 2> 374053 INFO (qtp516410542-1588) [n:127.0.0.1:60566_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 374894 INFO (org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5cc2157e) [ ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:40732 to delete [blk_1073741827_1003]
[junit4] 2> 375619 INFO (OverseerCollectionConfigSetProcessor-96914350840610826-127.0.0.1:60566_solr-n_0000000000) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 377894 INFO (org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor@5cc2157e) [ ] BlockStateChange BLOCK* BlockManager: ask 127.0.0.1:42118 to delete [blk_1073741827_1003]
[junit4] 2> 404057 ERROR (qtp516410542-1588) [n:127.0.0.1:60566_solr ] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE with timeout=30
[junit4] 2> 404057 INFO (qtp516410542-1588) [n:127.0.0.1:60566_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=halfcollection&action=CREATE&numShards=2&createNodeSet=127.0.0.1:48954_solr,127.0.0.1:40953_solr&wt=javabin&version=2} status=0 QTime=32448
[junit4] 2> 404058 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreateShouldFailOnExistingCore-seed#[F6C1D2403434BA00]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateShouldFailOnExistingCore
[junit4] 2> 404141 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[F6C1D2403434BA00]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMissingRequiredParameters
[junit4] 2> 404143 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[F6C1D2403434BA00]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 404148 INFO (qtp516410542-1582) [n:127.0.0.1:60566_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=halfcollectionblocker2&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 404152 INFO (OverseerThreadFactory-656-thread-3-processing-n:127.0.0.1:60566_solr) [n:127.0.0.1:60566_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 404164 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.c.SolrCore [halfcollection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@7eed0fd6
[junit4] 2> 404193 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollectionblocker2/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004/index
[junit4] 2> 404194 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollectionblocker2/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004/snapshot_metadata
[junit4] 2> 404194 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:53630/data/halfcollectionblocker2/core_node1/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-004
[junit4] 2> 404207 INFO (IPC Server handler 6 on 53630) [ ] BlockStateChange BLOCK* addToInvalidates: blk_1073741826_1002 127.0.0.1:40732 127.0.0.1:42118
[junit4] 2> 404213 INFO (qtp89624268-1557) [n:127.0.0.1:40953_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=halfcollection_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=49
[junit4] 2> 404896 INFO
[...truncated too long message...]
]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 555172 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53630) [ ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-575310218-10.41.0.5-1478795638828 (Datanode Uuid 6c5428c7-2126-40d8-82f5-5abb1bf54001) service to localhost/127.0.0.1:53630 interrupted
[junit4] 2> 555172 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53630) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-575310218-10.41.0.5-1478795638828 (Datanode Uuid 6c5428c7-2126-40d8-82f5-5abb1bf54001) service to localhost/127.0.0.1:53630
[junit4] 2> 555236 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 555240 ERROR (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
[junit4] 2> java.lang.reflect.InvocationTargetException
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 555254 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47793 47793
[junit4] 2> 555255 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F6C1D2403434BA00]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F6C1D2403434BA00-001
[junit4] 2> Nov 10, 2016 4:37:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1097, maxMBSortInHeap=5.053770508665871, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=en-CA, timezone=Brazil/East
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=5,free=388903744,total=493355008
[junit4] 2> NOTE: All tests run in this JVM: [TestTestInjection, TestRandomFaceting, SolrRequestParserTest, CurrencyFieldOpenExchangeTest, TestShortCircuitedRequests, MultiThreadedOCPTest, SolrCLIZkUtilsTest, TestSerializedLuceneMatchVersion, TestCloudPivotFacet, TestSolrCloudSnapshots, SolrCoreTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [58/651 (1!)] on J0 in 217.46s, 20 tests, 1 failure, 2 errors <<< FAILURES!
[...truncated 62569 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 194 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/194/
4 tests failed.
FAILED: org.apache.solr.core.TestCoreContainer.testCoreInitFailuresFromEmptyContainer
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([CC32D335FB3C47BB]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestCoreContainer
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([CC32D335FB3C47BB]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([CC32D335FB3C47BB:E14C9C6F0C18FDB7]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={ "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([CC32D335FB3C47BB:8447A781FD0F682E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11967 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/init-core-data-001
[junit4] 2> 1312199 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1312200 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001
[junit4] 2> 1312200 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1312201 INFO (Thread-3603) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1312201 INFO (Thread-3603) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1312301 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:34638
[junit4] 2> 1312309 INFO (jetty-launcher-876-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1312309 INFO (jetty-launcher-876-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1312310 INFO (jetty-launcher-876-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1312310 INFO (jetty-launcher-876-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1312324 INFO (jetty-launcher-876-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b6c0534{/solr,null,AVAILABLE}
[junit4] 2> 1312325 INFO (jetty-launcher-876-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3924e095{/solr,null,AVAILABLE}
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@65ffd482{SSL,[ssl, http/1.1]}{127.0.0.1:35620}
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.e.j.s.Server Started @1316358ms
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@1ca35a0b{SSL,[ssl, http/1.1]}{127.0.0.1:51690}
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35620}
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.e.j.s.Server Started @1316358ms
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51690}
[junit4] 2> 1312327 ERROR (jetty-launcher-876-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1312327 ERROR (jetty-launcher-876-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1312327 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1312328 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T17:27:14.939Z
[junit4] 2> 1312328 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T17:27:14.939Z
[junit4] 2> 1312338 INFO (jetty-launcher-876-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62d0c741{/solr,null,AVAILABLE}
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@6db3d17{SSL,[ssl, http/1.1]}{127.0.0.1:48116}
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.e.j.s.Server Started @1316371ms
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48116}
[junit4] 2> 1312340 ERROR (jetty-launcher-876-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1312340 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T17:27:14.951Z
[junit4] 2> 1312342 INFO (jetty-launcher-876-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20186f0e{/solr,null,AVAILABLE}
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@6cab192{SSL,[ssl, http/1.1]}{127.0.0.1:32944}
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.e.j.s.Server Started @1316375ms
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32944}
[junit4] 2> 1312344 ERROR (jetty-launcher-876-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1312344 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T17:27:14.955Z
[junit4] 2> 1312369 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1312375 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1312379 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1312380 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1312385 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1312386 INFO (jetty-launcher-876-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34638/solr
[junit4] 2> 1312387 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1312394 INFO (jetty-launcher-876-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34638/solr
[junit4] 2> 1312396 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1312396 INFO (jetty-launcher-876-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34638/solr
[junit4] 2> 1312397 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1312407 INFO (jetty-launcher-876-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34638/solr
[junit4] 2> 1312423 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1584a21207d0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1312434 INFO (jetty-launcher-876-thread-1) [n:127.0.0.1:51690_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51690_solr
[junit4] 2> 1312435 INFO (jetty-launcher-876-thread-1) [n:127.0.0.1:51690_solr ] o.a.s.c.Overseer Overseer (id=96908898127904776-127.0.0.1:51690_solr-n_0000000000) starting
[junit4] 2> 1312454 INFO (jetty-launcher-876-thread-3) [n:127.0.0.1:32944_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32944_solr
[junit4] 2> 1312459 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1312466 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1312466 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1312467 INFO (zkCallback-893-thread-1-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1312468 INFO (jetty-launcher-876-thread-2) [n:127.0.0.1:48116_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48116_solr
[junit4] 2> 1312468 INFO (jetty-launcher-876-thread-4) [n:127.0.0.1:35620_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35620_solr
[junit4] 2> 1312468 INFO (jetty-launcher-876-thread-3) [n:127.0.0.1:32944_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node3/.
[junit4] 2> 1312470 INFO (jetty-launcher-876-thread-1) [n:127.0.0.1:51690_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51690_solr
[junit4] 2> 1312470 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1312474 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1312477 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1312479 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1312480 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1312480 INFO (OverseerStateUpdate-96908898127904776-127.0.0.1:51690_solr-n_0000000000) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 1312482 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1312482 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1312482 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1312491 INFO (jetty-launcher-876-thread-4) [n:127.0.0.1:35620_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node4/.
[junit4] 2> 1312493 INFO (jetty-launcher-876-thread-1) [n:127.0.0.1:51690_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node1/.
[junit4] 2> 1312494 INFO (zkCallback-893-thread-2-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1312495 INFO (jetty-launcher-876-thread-2) [n:127.0.0.1:48116_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node2/.
[junit4] 2> 1312501 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1312541 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1312546 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1312549 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1312561 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_46742_hdfs____.acthk3/webapp
[junit4] 2> 1312972 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46742
[junit4] 2> 1313097 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1313104 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1313123 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60641_datanode____r4xcio/webapp
[junit4] 2> 1313560 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60641
[junit4] 2> 1313651 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1313654 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1313673 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_48159_datanode____r3zsoo/webapp
[junit4] 2> 1313837 INFO (IPC Server handler 3 on 47963) [ ] BlockStateChange BLOCK* processReport: from storage DS-d17c03e3-bdd0-42e9-aa7c-ac38003eacc5 node DatanodeRegistration(127.0.0.1:54270, datanodeUuid=44a353e9-9059-454a-96bc-eee8543815db, infoPort=35513, infoSecurePort=0, ipcPort=40672, storageInfo=lv=-56;cid=testClusterID;nsid=509662527;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1313837 INFO (IPC Server handler 3 on 47963) [ ] BlockStateChange BLOCK* processReport: from storage DS-78b738d3-cd65-45ad-ac56-135081a4cc18 node DatanodeRegistration(127.0.0.1:54270, datanodeUuid=44a353e9-9059-454a-96bc-eee8543815db, infoPort=35513, infoSecurePort=0, ipcPort=40672, storageInfo=lv=-56;cid=testClusterID;nsid=509662527;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1314149 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[CC32D335FB3C47BB]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:48159
[junit4] 2> 1314302 INFO (IPC Server handler 4 on 47963) [ ] BlockStateChange BLOCK* processReport: from storage DS-224f6f47-1aa3-4ffb-946e-4359cb22dde5 node DatanodeRegistration(127.0.0.1:35172, datanodeUuid=7a4db8a7-a6f6-424b-a7c4-f2d26d695cfa, infoPort=58969, infoSecurePort=0, ipcPort=48636, storageInfo=lv=-56;cid=testClusterID;nsid=509662527;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1314302 INFO (IPC Server handler 4 on 47963) [ ] BlockStateChange BLOCK* processReport: from storage DS-9cde1523-c0ec-4abd-9cfe-acd5e8a8c992 node DatanodeRegistration(127.0.0.1:35172, datanodeUuid=7a4db8a7-a6f6-424b-a7c4-f2d26d695cfa, infoPort=58969, infoSecurePort=0, ipcPort=48636, storageInfo=lv=-56;cid=testClusterID;nsid=509662527;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1314389 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTooManyReplicas
[junit4] 2> 1314390 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[CC32D335FB3C47BB]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1314455 INFO (qtp1486891540-7896) [n:127.0.0.1:35620_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1314459 INFO (OverseerThreadFactory-2871-thread-1-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.CreateCollectionCmd Create collection collection
[junit4] 2> 1314459 WARN (OverseerThreadFactory-2871-thread-1-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.CreateCollectionCmd Specified replicationFactor of 10 on collection collection is higher than or equal to the number of Solr instances currently live or live and part of your createNodeSet(4). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 1314459 ERROR (OverseerThreadFactory-2871-thread-1-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection operation: create failed:org.apache.solr.common.SolrException: Cannot create collection collection. Value of maxShardsPerNode is 1, and the number of nodes currently live or live and part of your createNodeSet is 4. This allows a maximum of 4 to be created. Value of numShards is 2 and value of replicationFactor is 10. This requires 20 shards to be created (higher than the allowed number)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:148)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:224)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:463)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1314464 INFO (qtp1486891540-7896) [n:127.0.0.1:35620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=10&collection.configName=conf&name=collection&action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=8
[junit4] 2> 1314466 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testTooManyReplicas-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTooManyReplicas
[junit4] 2> 1314550 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCoresAreDistributedAcrossNodes
[junit4] 2> 1314551 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[CC32D335FB3C47BB]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1314554 INFO (qtp1486891540-7897) [n:127.0.0.1:35620_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=nodes_used_collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1314557 INFO (OverseerThreadFactory-2871-thread-2-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.CreateCollectionCmd Create collection nodes_used_collection
[junit4] 2> 1314557 INFO (OverseerCollectionConfigSetProcessor-96908898127904776-127.0.0.1:51690_solr-n_0000000000) [n:127.0.0.1:51690_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1314704 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 1314731 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 1314744 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 1314746 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 1314851 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1314851 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1314851 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1314851 INFO (zkCallback-893-thread-1-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1315744 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1315747 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1315754 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1315788 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica1] Schema name=minimal
[junit4] 2> 1315792 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica2] Schema name=minimal
[junit4] 2> 1315795 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1315797 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1315797 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1315798 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 1315799 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 1315806 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica2] Schema name=minimal
[junit4] 2> 1315809 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1315811 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47963/data
[junit4] 2> 1315811 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1315811 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node1/nodes_used_collection_shard2_replica1], dataDir=[hdfs://localhost:47963/data/nodes_used_collection/core_node3/data/]
[junit4] 2> 1315811 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node3/data/snapshot_metadata
[junit4] 2> 1315812 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica1] Schema name=minimal
[junit4] 2> 1315814 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1315815 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 1315816 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47963/data
[junit4] 2> 1315816 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1315816 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node3/nodes_used_collection_shard2_replica2], dataDir=[hdfs://localhost:47963/data/nodes_used_collection/core_node4/data/]
[junit4] 2> 1315816 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node4/data/snapshot_metadata
[junit4] 2> 1315819 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 1315819 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47963/data
[junit4] 2> 1315819 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1315819 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node2/nodes_used_collection_shard1_replica1], dataDir=[hdfs://localhost:47963/data/nodes_used_collection/core_node2/data/]
[junit4] 2> 1315819 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node2/data/snapshot_metadata
[junit4] 2> 1315820 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:47963/data
[junit4] 2> 1315820 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1315820 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_CC32D335FB3C47BB-001/tempDir-001/node4/nodes_used_collection_shard1_replica2], dataDir=[hdfs://localhost:47963/data/nodes_used_collection/core_node1/data/]
[junit4] 2> 1315820 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1315828 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315828 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1315835 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315835 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1315838 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315838 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1315839 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315839 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1316041 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316043 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316045 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316046 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316046 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node4/data
[junit4] 2> 1316047 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node1/data
[junit4] 2> 1316051 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node2/data
[junit4] 2> 1316054 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node3/data
[junit4] 2> 1316074 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node4/data/index
[junit4] 2> 1316078 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node3/data/index
[junit4] 2> 1316085 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node1/data/index
[junit4] 2> 1316088 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1316088 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1316091 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1316092 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1316095 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1316095 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1316098 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316099 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316102 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:47963/data/nodes_used_collection/core_node2/data/index
[junit4] 2> 1316107 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316112 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1316112 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 1316127 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1316245 INFO (IPC Server handler 1 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35172 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d17c03e3-bdd0-42e9-aa7c-ac38003eacc5:NORMAL:127.0.0.1:54270|RBW], ReplicaUC[[DISK]DS-224f6f47-1aa3-4ffb-946e-4359cb22dde5:NORMAL:127.0.0.1:35172|FINALIZED]]} size 0
[junit4] 2> 1316247 INFO (IPC Server handler 8 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54270 is added to blk_1073741826_1002 size 71
[junit4] 2> 1316250 INFO (IPC Server handler 0 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54270 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-9cde1523-c0ec-4abd-9cfe-acd5e8a8c992:NORMAL:127.0.0.1:35172|RBW], ReplicaUC[[DISK]DS-78b738d3-cd65-45ad-ac56-135081a4cc18:NORMAL:127.0.0.1:54270|RBW]]} size 0
[junit4] 2> 1316250 INFO (IPC Server handler 5 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35172 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-9cde1523-c0ec-4abd-9cfe-acd5e8a8c992:NORMAL:127.0.0.1:35172|RBW], ReplicaUC[[DISK]DS-78b738d3-cd65-45ad-ac56-135081a4cc18:NORMAL:127.0.0.1:54270|RBW]]} size 0
[junit4] 2> 1316250 INFO (IPC Server handler 5 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35172 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-9cde1523-c0ec-4abd-9cfe-acd5e8a8c992:NORMAL:127.0.0.1:35172|RBW], ReplicaUC[[DISK]DS-78b738d3-cd65-45ad-ac56-135081a4cc18:NORMAL:127.0.0.1:54270|RBW]]} size 0
[junit4] 2> 1316251 INFO (IPC Server handler 6 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54270 is added to blk_1073741825_1001 size 71
[junit4] 2> 1316305 INFO (IPC Server handler 0 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35172 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d17c03e3-bdd0-42e9-aa7c-ac38003eacc5:NORMAL:127.0.0.1:54270|RBW], ReplicaUC[[DISK]DS-224f6f47-1aa3-4ffb-946e-4359cb22dde5:NORMAL:127.0.0.1:35172|RBW]]} size 0
[junit4] 2> 1316305 INFO (IPC Server handler 5 on 47963) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54270 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d17c03e3-bdd0-42e9-aa7c-ac38003eacc5:NORMAL:127.0.0.1:54270|RBW], ReplicaUC[[DISK]DS-224f6f47-1aa3-4ffb-946e-4359cb22dde5:NORMAL:127.0.0.1:35172|RBW]]} size 0
[junit4] 2> 1316310 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1316310 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1316310 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1316314 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1316314 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1316314 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1316318 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1316318 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1316318 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1316332 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1316332 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1316343 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1316343 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1316351 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1316351 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1316353 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1316353 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1316353 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 1316366 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@d03be0d[nodes_used_collection_shard1_replica2] main]
[junit4] 2> 1316367 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1316367 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1316367 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1316371 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550542374417989632
[junit4] 2> 1316373 INFO (searcherExecutor-2879-thread-1-processing-n:127.0.0.1:35620_solr x:nodes_used_collection_shard1_replica2 s:shard1 c:nodes_used_collection r:core_node1) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@d03be0d[nodes_used_collection_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1316374 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@145aaca4[nodes_used_collection_shard2_replica1] main]
[junit4] 2> 1316376 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1316376 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1316377 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1316377 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550542374424281088
[junit4] 2> 1316377 INFO (searcherExecutor-2878-thread-1-processing-n:127.0.0.1:51690_solr x:nodes_used_collection_shard2_replica1 s:shard2 c:nodes_used_collection r:core_node3) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@145aaca4[nodes_used_collection_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1316390 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1316390 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1316430 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1316431 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316431 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316431 INFO (zkCallback-893-thread-2-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316431 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316435 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 1316436 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4028db32[nodes_used_collection_shard2_replica2] main]
[junit4] 2> 1316437 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1316437 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1316437 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1316438 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550542374488244224
[junit4] 2> 1316438 INFO (searcherExecutor-2880-thread-1-processing-n:127.0.0.1:32944_solr x:nodes_used_collection_shard2_replica2 s:shard2 c:nodes_used_collection r:core_node4) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@4028db32[nodes_used_collection_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1316447 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@90cf88[nodes_used_collection_shard1_replica1] main]
[junit4] 2> 1316448 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1316449 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1316449 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1316449 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1550542374499778560
[junit4] 2> 1316451 INFO (searcherExecutor-2881-thread-1-processing-n:127.0.0.1:48116_solr x:nodes_used_collection_shard1_replica1 s:shard1 c:nodes_used_collection r:core_node2) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@90cf88[nodes_used_collection_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1316538 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316538 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316539 INFO (zkCallback-893-thread-2-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316539 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1316931 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1316931 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1316931 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:35620/solr/nodes_used_collection_shard1_replica2/
[junit4] 2> 1316931 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard1_replica2 url=https://127.0.0.1:35620/solr START replicas=[https://127.0.0.1:48116/solr/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4] 2> 1316931 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard1_replica2 url=https://127.0.0.1:35620/solr DONE. We have no versions. sync failed.
[junit4] 2> 1316936 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1316936 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1316936 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51690/solr/nodes_used_collection_shard2_replica1/
[junit4] 2> 1316936 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard2_replica1 url=https://127.0.0.1:51690/solr START replicas=[https://127.0.0.1:32944/solr/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4] 2> 1316936 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=nodes_used_collection_shard2_replica1 url=https://127.0.0.1:51690/solr DONE. We have no versions. sync failed.
[junit4] 2> 1316938 INFO (qtp920440528-7901) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.c.S.Request [nodes_used_collection_shard1_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1316939 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1316939 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1316941 INFO (qtp2011841402-7884) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.c.S.Request [nodes_used_collection_shard2_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1316942 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1316942 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1316947 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51690/solr/nodes_used_collection_shard2_replica1/ shard2
[junit4] 2> 1316948 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:35620/solr/nodes_used_collection_shard1_replica2/ shard1
[junit4] 2> 1317055 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317055 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317055 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317058 INFO (zkCallback-893-thread-3-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317099 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1317099 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1317101 INFO (qtp1486891540-7894) [n:127.0.0.1:35620_solr c:nodes_used_collection s:shard1 r:core_node1 x:nodes_used_collection_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2} status=0 QTime=2398
[junit4] 2> 1317103 INFO (qtp146862523-7902) [n:127.0.0.1:51690_solr c:nodes_used_collection s:shard2 r:core_node3 x:nodes_used_collection_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2} status=0 QTime=2358
[junit4] 2> 1317207 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317207 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317208 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317208 INFO (zkCallback-893-thread-3-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317449 INFO (qtp2011841402-7916) [n:127.0.0.1:32944_solr c:nodes_used_collection s:shard2 r:core_node4 x:nodes_used_collection_shard2_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2} status=0 QTime=2703
[junit4] 2> 1317455 INFO (qtp920440528-7908) [n:127.0.0.1:48116_solr c:nodes_used_collection s:shard1 r:core_node2 x:nodes_used_collection_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2} status=0 QTime=2724
[junit4] 2> 1317460 INFO (qtp1486891540-7897) [n:127.0.0.1:35620_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1317560 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:32944_solr) [n:127.0.0.1:32944_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317560 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:48116_solr) [n:127.0.0.1:48116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317560 INFO (zkCallback-893-thread-2-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1317560 INFO (zkCallback-898-thread-1-processing-n:127.0.0.1:35620_solr) [n:127.0.0.1:35620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 1318460 INFO (qtp1486891540-7897) [n:127.0.0.1:35620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=nodes_used_collection&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3905
[junit4] 2> 1318461 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCoresAreDistributedAcrossNodes
[junit4] 2> 1318534 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreationAndDeletion
[junit4] 2> 1318536 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCreationAndDeletion-seed#[CC32D335FB3C47BB]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1318539 INFO (qtp1486891540-7898) [n:127.0.0.1:35620_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=nodes_used_collection&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1318541 INFO (OverseerThreadFactory-2871-thread-3-processing-n:127.0.0.1:51690_solr) [n:127.0.0.1:51690_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1318547 INFO (OverseerCollectionConfigSetProcessor-96908898127904776-127.0.0.1:51690_solr-n_0000000000) [n:127.0.0.1:51690_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1318547 INFO (qtp1486891540-7896) [n:127.0.0.1:35620_solr ] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@1f1df52b
[junit4] 2> 1318548 INFO (qtp146862523-7903) [n:127.0.0.1:51690_solr ] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@5691d403
[junit4] 2> 1318550 INFO (qtp920440528-7909) [n:127.0.0.1:48116_solr ] o.a.s.c.SolrCore [nodes_used_collection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@ee45dd3
[junit4] 2> 1318552 INFO (qtp2011841402-7917) [n:127.0.0.1:32944_solr ] o.a.s.c.SolrCore [nodes_used_collection_shard2_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@7678c71e
[junit4] 2> 1318558 INFO (qtp920440528-7909) [n:127.0.0.1:48116_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node2/data/snapshot_metadata
[junit4] 2> 1318559 INFO (qtp920440528-7909) [n:127.0.0.1:48116_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node2/data
[junit4] 2> 1318561 INFO (qtp920440528-7909) [n:127.0.0.1:48116_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node2/data/index
[junit4] 2> 1318568 INFO (qtp146862523-7903) [n:127.0.0.1:51690_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node3/data/snapshot_metadata
[junit4] 2> 1318568 INFO (qtp2011841402-7917) [n:127.0.0.1:32944_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node4/data/index
[junit4] 2> 1318569 INFO (qtp146862523-7903) [n:127.0.0.1:51690_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node3/data/index
[junit4] 2> 1318569 INFO (qtp146862523-7903) [n:127.0.0.1:51690_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node3/data
[junit4] 2> 1318574 INFO (qtp1486891540-7896) [n:127.0.0.1:35620_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1318575 INFO (qtp2011841402-7917) [n:127.0.0.1:32944_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node4/data/snapshot_metadata
[junit4] 2> 1318575 INFO (qtp2011841402-7917) [n:127.0.0.1:32944_solr ] o.a.s.s.h.HdfsDirectory Closing hdfs directory hdfs://localhost:47963/data/nodes_used_collection/core_node4/data
[junit4] 2> 1318579 INFO (qtp1486891540-7896) [n:127.0.0.1:35620_solr ] o.a.s.s.h.HdfsDirectory Closing h
[...truncated too long message...]
at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2) Thread[id=40629, name=solr-idle-connections-evictor, state=TIMED_WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.update.UpdateShardHandler$IdleConnectionsEvictor$1.run(UpdateShardHandler.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3) Thread[id=40630, name=Thread-35572, state=WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:1274)
[junit4] 2> 4) Thread[id=40628, name=solr-idle-connections-evictor, state=TIMED_WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.update.UpdateShardHandler$IdleConnectionsEvictor$1.run(UpdateShardHandler.java:187)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> ??? 09, 2016 4:39:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=40627, name=TEST-TestCoreContainer.testCoreInitFailuresFromEmptyContainer-seed#[CC32D335FB3C47BB], state=WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> 2) Thread[id=40629, name=solr-idle-connections-evictor, state=TIMED_WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> 3) Thread[id=40630, name=Thread-35572, state=WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> 4) Thread[id=40628, name=solr-idle-connections-evictor, state=TIMED_WAITING, group=TGRP-TestCoreContainer]
[junit4] 2> 9252985 INFO (TEST-TestCoreContainer.testCoreInitFailuresFromEmptyContainer-seed#[CC32D335FB3C47BB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCoreInitFailuresFromEmptyContainer
[junit4] 2> 9252986 INFO (SUITE-TestCoreContainer-seed#[CC32D335FB3C47BB]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> Nov 09, 2016 7:39:35 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3, maxDocsPerChunk=10, blockSize=648), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3, blockSize=648)), sim=ClassicSimilarity, locale=sr, timezone=America/Fortaleza
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=295178848,total=529006592
[junit4] 2> NOTE: All tests run in this JVM: [TestSmileRequest, TestSystemIdResolver, TestSolrIndexConfig, TestBackupRepositoryFactory, RestartWhileUpdatingTest, TestRequestStatusCollectionAPI, TestDocBasedVersionConstraints, TestJsonFacets, TestCloudPseudoReturnFields, TestSQLHandlerNonCloud, ChaosMonkeySafeLeaderTest, TestManagedStopFilterFactory, TestAuthorizationFramework, TestTrie, TestPartialUpdateDeduplication, TestLuceneMatchVersion, SimplePostToolTest, BlockDirectoryTest, DeleteNodeTest, TestCollectionAPI, TestHdfsCloudBackupRestore, AnalyticsQueryTest, DistributedMLTComponentTest, TestScoreJoinQPNoScore, DistributedTermsComponentTest, SaslZkACLProviderTest, TestGraphTermsQParserPlugin, LoggingHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, OpenCloseCoreStressTest, SoftAutoCommitTest, SearchHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest, DocumentBuilderTest, TestSimpleTrackingShardHandler, RAMDirectoryFactoryTest, SharedFSAutoReplicaFailoverUtilsTest, CloudMLTQParserTest, UUIDUpdateProcessorFallbackTest, HdfsNNFailoverTest, TestBulkSchemaAPI, TestSlowCompositeReaderWrapper, TestQuerySenderListener, SpellingQueryConverterTest, SliceStateTest, TestPerFieldSimilarityWithDefaultOverride, TestStandardQParsers, TestDistribIDF, DocValuesMissingTest, RulesTest, TestCustomDocTransformer, FileUtilsTest, TestRebalanceLeaders, TestSweetSpotSimilarityFactory, HdfsTlogReplayBufferedWhileIndexingTest, IndexSchemaRuntimeFieldTest, JsonLoaderTest, DeleteStatusTest, TestSortByMinMaxFunction, TestQueryUtils, TestFieldCacheReopen, RemoteQueryErrorTest, ExitableDirectoryReaderTest, TestLMDirichletSimilarityFactory, CoreAdminCreateDiscoverTest, BasicFunctionalityTest, TestSolrDynamicMBean, TestDFISimilarityFactory, TestAnalyzeInfixSuggestions, DistributedIntervalFacetingTest, TestSurroundQueryParser, HdfsRestartWhileUpdatingTest, TestFieldCacheSortRandom, TestCoreContainer]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCoreContainer -Dtests.seed=CC32D335FB3C47BB -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr -Dtests.timezone=America/Fortaleza -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | TestCoreContainer (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([CC32D335FB3C47BB]:0)
[junit4] Completed [618/651 (2!)] on J1 in 7220.25s, 1 test, 2 errors <<< FAILURES!
[...truncated 60755 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 193 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/193/
3 tests failed.
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1478652300740,generation=2,filelist=[_3.cfe, _3.cfs, _3.si, _4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1478652300740,generation=2,filelist=[_3.cfe, _3.cfs, _3.si, _4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1478652300740,generation=3,filelist=[_4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _7.doc, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7.tim, _7.tip, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1478652300740,generation=2,filelist=[_3.cfe, _3.cfs, _3.si, _4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1478652300740,generation=2,filelist=[_3.cfe, _3.cfs, _3.si, _4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1478652300740,generation=3,filelist=[_4.doc, _4.fdt, _4.fdx, _4.fnm, _4.nvd, _4.nvm, _4.si, _4.tim, _4.tip, _7.doc, _7.fdt, _7.fdx, _7.fnm, _7.nvd, _7.nvm, _7.si, _7.tim, _7.tip, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([C561324EA9830CD7:E0B6297ED9CB02D4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/11)={ "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-bfffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"c0000000-ffffffff", "state":"active", "replicas":{}}, "shard3":{ "range":"0-3fffffff", "state":"active", "replicas":{}}, "shard4":{ "range":"40000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"4", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/11)={
"replicationFactor":"3",
"shards":{
"shard1":{
"range":"80000000-bfffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"0-3fffffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"4",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([C561324EA9830CD7:8D1446FAAFB02342]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([C561324EA9830CD7:E81F7D145EA7B6DB]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12473 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/init-core-data-001
[junit4] 2> 1866262 INFO (SUITE-TestReplicationHandler-seed#[C561324EA9830CD7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 1866264 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestHandlerPathUnchanged
[junit4] 2> 1866265 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/collection1
[junit4] 2> 1866308 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1866379 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d258d73{/solr,null,AVAILABLE}
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Started ServerConnector@49caceae{HTTP/1.1,[http/1.1]}{127.0.0.1:48856}
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server Started @1870956ms
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=48856}
[junit4] 2> 1866380 ERROR (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1866380 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T00:37:18.606Z
[junit4] 2> 1866396 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/solr.xml
[junit4] 2> 1866415 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1866433 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/.
[junit4] 2> 1866433 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1866469 WARN (coreLoadExecutor-4555-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1866470 INFO (coreLoadExecutor-4555-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1866479 INFO (coreLoadExecutor-4555-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1866673 INFO (coreLoadExecutor-4555-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1866677 INFO (coreLoadExecutor-4555-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/./collection1
[junit4] 2> 1866677 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/./collection1/data/]
[junit4] 2> 1866678 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=440812063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1866685 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1866685 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1866686 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.599609375, noCFSRatio=0.6405649621294429]
[junit4] 2> 1866686 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@135b8fef[collection1] main]
[junit4] 2> 1866686 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-001/collection1/conf
[junit4] 2> 1866687 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 1866687 INFO (coreLoadExecutor-4555-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1866687 INFO (searcherExecutor-4556-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@135b8fef[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1866965 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/collection1
[junit4] 2> 1866968 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1866988 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@144c2496{/solr,null,AVAILABLE}
[junit4] 2> 1866988 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Started ServerConnector@1cb36ece{HTTP/1.1,[http/1.1]}{127.0.0.1:44889}
[junit4] 2> 1866989 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server Started @1871564ms
[junit4] 2> 1866989 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=44889}
[junit4] 2> 1866989 ERROR (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1866990 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1866990 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1866990 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1866991 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T00:37:19.216Z
[junit4] 2> 1866991 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/solr.xml
[junit4] 2> 1867000 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1867002 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/.
[junit4] 2> 1867002 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1867013 WARN (coreLoadExecutor-4565-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1867015 INFO (coreLoadExecutor-4565-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1867031 INFO (coreLoadExecutor-4565-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1867036 INFO (coreLoadExecutor-4565-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1867038 INFO (coreLoadExecutor-4565-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/./collection1
[junit4] 2> 1867040 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/./collection1/data/]
[junit4] 2> 1867044 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=440812063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1867050 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1867050 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1867051 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.599609375, noCFSRatio=0.6405649621294429]
[junit4] 2> 1867051 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@840de3a[collection1] main]
[junit4] 2> 1867051 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-002/collection1/conf
[junit4] 2> 1867053 INFO (coreLoadExecutor-4565-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 1867054 INFO (searcherExecutor-4566-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@840de3a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1867505 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTestHandlerPathUnchanged
[junit4] 2> 1867510 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@49caceae{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1867510 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=130693231
[junit4] 2> 1867512 INFO (coreCloseExecutor-4571-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1281a98
[junit4] 2> 1867548 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4d258d73{/solr,null,UNAVAILABLE}
[junit4] 2> 1867549 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@1cb36ece{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1867549 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1421192957
[junit4] 2> 1867550 INFO (coreCloseExecutor-4573-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5fff3545
[junit4] 2> 1867552 INFO (TEST-TestReplicationHandler.doTestHandlerPathUnchanged-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@144c2496{/solr,null,UNAVAILABLE}
[junit4] 2> 1867555 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestStopPoll
[junit4] 2> 1867555 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/collection1
[junit4] 2> 1867560 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1867576 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2918f2e8{/solr,null,AVAILABLE}
[junit4] 2> 1867576 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Started ServerConnector@16672cc9{HTTP/1.1,[http/1.1]}{127.0.0.1:56412}
[junit4] 2> 1867576 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server Started @1872152ms
[junit4] 2> 1867576 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=56412}
[junit4] 2> 1867577 ERROR (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1867578 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1867578 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1867578 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1867578 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T00:37:19.804Z
[junit4] 2> 1867578 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/solr.xml
[junit4] 2> 1867639 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1867657 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/.
[junit4] 2> 1867657 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1867701 WARN (coreLoadExecutor-4580-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1867703 INFO (coreLoadExecutor-4580-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1867735 INFO (coreLoadExecutor-4580-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1867737 INFO (coreLoadExecutor-4580-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1867738 INFO (coreLoadExecutor-4580-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/./collection1
[junit4] 2> 1867738 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/./collection1/data/]
[junit4] 2> 1867739 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=440812063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1867745 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1867745 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1867745 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.599609375, noCFSRatio=0.6405649621294429]
[junit4] 2> 1867746 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6bb2082d[collection1] main]
[junit4] 2> 1867746 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-003/collection1/conf
[junit4] 2> 1867746 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 1867746 INFO (coreLoadExecutor-4580-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1867747 INFO (searcherExecutor-4581-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6bb2082d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1868178 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/collection1
[junit4] 2> 1868181 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1868196 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33224e21{/solr,null,AVAILABLE}
[junit4] 2> 1868203 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.ServerConnector Started ServerConnector@31b917a7{HTTP/1.1,[http/1.1]}{127.0.0.1:50700}
[junit4] 2> 1868215 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.e.j.s.Server Started @1872791ms
[junit4] 2> 1868215 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/collection1/data, hostContext=/solr, hostPort=50700}
[junit4] 2> 1868219 ERROR (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1868220 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 1868220 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1868220 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1868220 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-09T00:37:20.446Z
[junit4] 2> 1868220 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/solr.xml
[junit4] 2> 1868251 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1868253 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/.
[junit4] 2> 1868253 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[C561324EA9830CD7]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1868271 WARN (coreLoadExecutor-4590-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1868272 INFO (coreLoadExecutor-4590-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1868289 INFO (coreLoadExecutor-4590-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1868292 INFO (coreLoadExecutor-4590-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1868300 INFO (coreLoadExecutor-4590-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/./collection1
[junit4] 2> 1868300 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/./collection1/data/]
[junit4] 2> 1868302 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=440812063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1868313 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1868313 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1868313 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=3, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.599609375, noCFSRatio=0.6405649621294429]
[junit4] 2> 1868313 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b49686[collection1] main]
[junit4] 2> 1868313 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_C561324EA9830CD7-001/solr-instance-004/collection1/conf
[junit4] 2> 1868314 INFO (coreLoadExecutor-4590-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 1868315 INFO (searcherExecutor-4591-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7b49686[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1868600 INFO (qtp1975375051-59321) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 1868601 INFO (indexFetcher-4595-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1868601 INFO (indexFetcher-4595-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1868601 INFO (indexFetcher-4595-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1868762 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1868766 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 1
[junit4] 2> 1868768 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 1868770 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 1868772 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
[junit4] 2> 1868774 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 1868776 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
[junit4] 2> 1868778 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
[junit4] 2> 1868779 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
[junit4] 2> 1868782 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
[junit4] 2> 1868783 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
[junit4] 2> 1868785 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
[junit4] 2> 1868787 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
[junit4] 2> 1868788 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
[junit4] 2> 1868790 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
[junit4] 2> 1868792 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
[junit4] 2> 1868794 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
[junit4] 2> 1868796 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
[junit4] 2> 1868797 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
[junit4] 2> 1868799 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
[junit4] 2> 1868801 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
[junit4] 2> 1868803 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
[junit4] 2> 1868805 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
[junit4] 2> 1868806 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
[junit4] 2> 1868808 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
[junit4] 2> 1868810 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
[junit4] 2> 1868812 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
[junit4] 2> 1868814 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
[junit4] 2> 1868815 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
[junit4] 2> 1868817 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
[junit4] 2> 1868819 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
[junit4] 2> 1868821 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
[junit4] 2> 1868823 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
[junit4] 2> 1868825 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
[junit4] 2> 1868827 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
[junit4] 2> 1868828 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
[junit4] 2> 1868830 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
[junit4] 2> 1868832 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
[junit4] 2> 1868835 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
[junit4] 2> 1868837 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
[junit4] 2> 1868839 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
[junit4] 2> 1868843 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
[junit4] 2> 1868845 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
[junit4] 2> 1868847 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
[junit4] 2> 1868849 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
[junit4] 2> 1868851 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
[junit4] 2> 1868853 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
[junit4] 2> 1868855 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
[junit4] 2> 1868857 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
[junit4] 2> 1868859 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
[junit4] 2> 1868861 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
[junit4] 2> 1868863 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
[junit4] 2> 1868865 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
[junit4] 2> 1868867 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
[junit4] 2> 1868869 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
[junit4] 2> 1868870 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
[junit4] 2> 1868874 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
[junit4] 2> 1868876 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
[junit4] 2> 1868878 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
[junit4] 2> 1868880 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
[junit4] 2> 1868882 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
[junit4] 2> 1868884 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
[junit4] 2> 1868886 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
[junit4] 2> 1868888 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
[junit4] 2> 1868890 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
[junit4] 2> 1868892 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
[junit4] 2> 1868894 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
[junit4] 2> 1868896 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
[junit4] 2> 1868899 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
[junit4] 2> 1868901 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
[junit4] 2> 1868903 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
[junit4] 2> 1868905 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
[junit4] 2> 1868907 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
[junit4] 2> 1868912 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
[junit4] 2> 1868917 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
[junit4] 2> 1868924 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
[junit4] 2> 1868946 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
[junit4] 2> 1868948 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
[junit4] 2> 1868950 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
[junit4] 2> 1868956 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
[junit4] 2> 1868959 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
[junit4] 2> 1868961 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
[junit4] 2> 1868970 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
[junit4] 2> 1868972 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
[junit4] 2> 1868974 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
[junit4] 2> 1868978 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
[junit4] 2> 1868980 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
[junit4] 2> 1868981 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
[junit4] 2> 1868983 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
[junit4] 2> 1868984 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
[junit4] 2> 1868987 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
[junit4] 2> 1868989 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
[junit4] 2> 1868991 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
[junit4] 2> 1868995 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
[junit4] 2> 1869012 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
[junit4] 2> 1869013 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
[junit4] 2> 1869015 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
[junit4] 2> 1869017 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
[junit4] 2> 1869018 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
[junit4] 2> 1869020 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
[junit4] 2> 1869021 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
[junit4] 2> 1869023 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
[junit4] 2> 1869028 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
[junit4] 2> 1869030 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
[junit4] 2> 1869031 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
[junit4] 2> 1869033 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
[junit4] 2> 1869035 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
[junit4] 2> 1869037 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
[junit4] 2> 1869039 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
[junit4] 2> 1869041 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
[junit4] 2> 1869043 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
[junit4] 2> 1869045 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
[junit4] 2> 1869047 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
[junit4] 2> 1869049 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
[junit4] 2> 1869050 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
[junit4] 2> 1869052 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
[junit4] 2> 1869054 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
[junit4] 2> 1869056 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
[junit4] 2> 1869058 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
[junit4] 2> 1869060 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
[junit4] 2> 1869063 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
[junit4] 2> 1869065 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
[junit4] 2> 1869068 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 0
[junit4] 2> 1869071 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
[junit4] 2> 1869072 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
[junit4] 2> 1869074 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
[junit4] 2> 1869077 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
[junit4] 2> 1869079 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
[junit4] 2> 1869082 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
[junit4] 2> 1869085 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
[junit4] 2> 1869087 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
[junit4] 2> 1869089 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
[junit4] 2> 1869091 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
[junit4] 2> 1869093 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
[junit4] 2> 1869095 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
[junit4] 2> 1869098 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
[junit4] 2> 1869100 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
[junit4] 2> 1869102 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
[junit4] 2> 1869105 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
[junit4] 2> 1869107 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
[junit4] 2> 1869108 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
[junit4] 2> 1869110 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
[junit4] 2> 1869112 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
[junit4] 2> 1869113 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
[junit4] 2> 1869131 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
[junit4] 2> 1869132 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
[junit4] 2> 1869136 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
[junit4] 2> 1869138 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
[junit4] 2> 1869139 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
[junit4] 2> 1869141 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
[junit4] 2> 1869142 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
[junit4] 2> 1869144 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
[junit4] 2> 1869146 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
[junit4] 2> 1869148 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
[junit4] 2> 1869149 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
[junit4] 2> 1869151 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
[junit4] 2> 1869153 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
[junit4] 2> 1869155 INFO (qtp1975375051-59325) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
[junit4] 2> 1869157 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
[junit4] 2> 1869159 INFO (qtp1975375051-59324) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
[junit4] 2> 1869164 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
[junit4] 2> 1869167 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
[junit4] 2> 1869169 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
[junit4] 2> 1869170 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
[junit4] 2> 1869172 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
[junit4] 2> 1869174 INFO (qtp1975375051-59319) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
[junit4] 2> 1869176 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 0
[junit4] 2> 1869191 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
[junit4] 2> 1869193 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
[junit4] 2> 1869195 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
[junit4] 2> 1869197 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
[junit4] 2> 1869199 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
[junit4] 2> 1869200 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
[junit4] 2> 1869202 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
[junit4] 2> 1869204 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
[junit4] 2> 1869206 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
[junit4] 2> 1869208 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
[junit4] 2> 1869209 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
[junit4] 2> 1869211 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
[junit4] 2> 1869213 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
[junit4] 2> 1869214 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
[junit4] 2> 1869216 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
[junit4] 2> 1869218 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
[junit4] 2> 1869219 INFO (qtp1975375051-59323) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
[junit4] 2> 1869221 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
[junit4] 2> 1869223 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
[junit4] 2> 1869224 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
[junit4] 2> 1869226 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
[junit4] 2> 1869228 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
[junit4] 2> 1869230 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
[junit4] 2> 1869231 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 0
[junit4] 2> 1869233 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
[junit4] 2> 1869235 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
[junit4] 2> 1869237 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
[junit4] 2> 1869238 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
[junit4] 2> 1869240 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
[junit4] 2> 1869241 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
[junit4] 2> 1869243 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
[junit4] 2> 1869244 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
[junit4] 2> 1869246 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[198]} 0 0
[junit4] 2> 1869248 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[199]} 0 0
[junit4] 2> 1869250 INFO (qtp1975375051-59322) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[200]} 0 0
[junit4] 2> 1869252 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[201]} 0 0
[junit4] 2> 1869262 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[202]} 0 0
[junit4] 2> 1869264 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[203]} 0 0
[junit4] 2> 1869266 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[204]} 0 0
[junit4] 2> 1869268 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[205]} 0 0
[junit4] 2> 1869270 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[206]} 0 0
[junit4] 2> 1869271 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[207]} 0 0
[junit4] 2> 1869273 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[208]} 0 0
[junit4] 2> 1869274 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[209]} 0 0
[junit4] 2> 1869276 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[210]} 0 0
[junit4] 2> 1869278 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[211]} 0 0
[junit4] 2> 1869279 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[212]} 0 0
[junit4] 2> 1869281 INFO (qtp1975375051-59318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[213]} 0 0
[junit4] 2> 1869285 INFO (qtp19753
[...truncated too long message...]
SystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 4702505 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C561324EA9830CD7]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41676 41676
[junit4] 2> 4702506 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[C561324EA9830CD7]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_C561324EA9830CD7-001
[junit4] 2> Nov 09, 2016 1:24:34 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=it-IT, timezone=Asia/Yerevan
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=180985936,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestSchemaResource, ConfigSetsAPITest, SimplePostToolTest, TestSizeLimitedDistributedMap, DocExpirationUpdateProcessorFactoryTest, ShardRoutingCustomTest, FileUtilsTest, SaslZkACLProviderTest, RemoteQueryErrorTest, TestUniqueKeyFieldResource, TestMacros, SolrCloudExampleTest, DeleteNodeTest, TestRecoveryHdfs, TestFieldTypeCollectionResource, TestRuleBasedAuthorizationPlugin, SuggestComponentTest, VMParamsZkACLAndCredentialsProvidersTest, PeerSyncWithIndexFingerprintCachingTest, TestMiniSolrCloudCluster, TestLeaderInitiatedRecoveryThread, TestReqParamsAPI, HighlighterMaxOffsetTest, DateRangeFieldTest, MoreLikeThisHandlerTest, TestSurroundQueryParser, TestSweetSpotSimilarityFactory, TestWordDelimiterFilterFactory, TestDistributedGrouping, SoftAutoCommitTest, TestCopyFieldCollectionResource, TermVectorComponentTest, TestJsonRequest, SpatialRPTFieldTypeTest, TestRTGBase, DistribJoinFromCollectionTest, TestCollectionAPI, TestJsonFacetRefinement, TestSuggestSpellingConverter, TestExactStatsCache, LoggingHandlerTest, TestTestInjection, TestSolrFieldCacheMBean, CSVRequestHandlerTest, PreAnalyzedUpdateProcessorTest, TestStressCloudBlindAtomicUpdates, LeaderInitiatedRecoveryOnShardRestartTest, TestTrieFacet, TestPerFieldSimilarityClassic, DistributedFacetExistsSmallTest, SolrIndexConfigTest, TestJettySolrRunner, TestFieldCacheWithThreads, SampleTest, TestReloadAndDeleteDocs, MergeStrategyTest, ClusterStateUpdateTest, TestConfigSetsAPIZkFailure, ConjunctionSolrSpellCheckerTest, TestQuerySenderListener, TestChildDocTransformer, SchemaVersionSpecificBehaviorTest, RulesTest, TestObjectReleaseTracker, TestRebalanceLeaders, SpellingQueryConverterTest, TestGeoJSONResponseWriter, TestQueryUtils, TestFileDictionaryLookup, SparseHLLTest, TestSolrIndexConfig, TestHashPartitioner, WordBreakSolrSpellCheckerTest, TestCursorMarkWithoutUniqueKey, HdfsRestartWhileUpdatingTest, CdcrReplicationHandlerTest, TestConfigSetProperties, BadCopyFieldTest, AtomicUpdatesTest, TestClassicSimilarityFactory, TestRestManager, TestCharFilters, TestFieldCacheSort, OverseerModifyCollectionTest, DistributedQueryComponentCustomSortTest, SolrIndexSplitterTest, MultiTermTest, TestSQLHandlerNonCloud, TestInfoStreamLogging, TestFiltering, CopyFieldTest, TestBM25SimilarityFactory, TestTolerantUpdateProcessorRandomCloud, TestHdfsUpdateLog, TestSystemIdResolver, ExitableDirectoryReaderTest, TestLMDirichletSimilarityFactory, ZkSolrClientTest, TestFastOutputStream, TestInitQParser, TestDistribDocBasedVersion, TestRequestStatusCollectionAPI, DirectSolrSpellCheckerTest, TestOnReconnectListenerSupport, TestNoOpRegenerator, OpenCloseCoreStressTest, AnalysisErrorHandlingTest, CustomCollectionTest, SystemInfoHandlerTest, LeaderElectionTest, ZkCLITest, AutoCommitTest, ZkControllerTest, TestRealTimeGet, TestJoin, TestReload, TestStressVersions, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, DistributedQueryElevationComponentTest, TestCoreDiscovery, SpellCheckCollatorTest, SuggesterFSTTest, DocValuesTest, SuggesterTest, TestStressLucene, TestTrie, SpatialFilterTest, SuggesterWFSTTest, PolyFieldTest, NoCacheHeaderTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, QueryEqualityTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, DistanceFunctionTest, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, TermsComponentTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, MBeansHandlerTest, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestComponentsName, SearchHandlerTest, TestBinaryResponseWriter, TestLFUCache, SOLR749Test, BadComponentTest, TestStressRecovery, PreAnalyzedFieldTest, DOMUtilTest, RAMDirectoryFactoryTest, TestSolrJ, TestLRUCache, SliceStateTest, CircularListTest, DistributedMLTComponentTest, CursorPagingTest, DistributedIntervalFacetingTest, AssignTest, AsyncCallRequestStatusResponseTest, CdcrReplicationDistributedZkTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [511/649 (2!)] on J1 in 214.11s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 61059 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 192 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/192/
5 tests failed.
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([150B8ABDB6F96C42:9D5FB567180501BA]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:199)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest.test
Error Message:
No registered leader was found after waiting for 10000ms , collection: collection1 slice: shard5
Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found after waiting for 10000ms , collection: collection1 slice: shard5
at __randomizedtesting.SeedInfo.seed([150B8ABDB6F96C42:9D5FB567180501BA]:0)
at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:747)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:427)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:332)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:990)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1478134222177,generation=2,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bm1.fdt, _bm1.fdx, _bm1.fnm, _bm1.nvd, _bm1.nvm, _bm1.si, _bm1_TestBloomFilteredLucenePostings_0.blm, _bm1_TestBloomFilteredLucenePostings_0.doc, _bm1_TestBloomFilteredLucenePostings_0.tim, _bm1_TestBloomFilteredLucenePostings_0.tip, _bm2.fdt, _bm2.fdx, _bm2.fnm, _bm2.nvd, _bm2.nvm, _bm2.si, _bm2_TestBloomFilteredLucenePostings_0.blm, _bm2_TestBloomFilteredLucenePostings_0.doc, _bm2_TestBloomFilteredLucenePostings_0.tim, _bm2_TestBloomFilteredLucenePostings_0.tip, _bm3.fdt, _bm3.fdx, _bm3.fnm, _bm3.nvd, _bm3.nvm, _bm3.si, _bm3_TestBloomFilteredLucenePostings_0.blm, _bm3_TestBloomFilteredLucenePostings_0.doc, _bm3_TestBloomFilteredLucenePostings_0.tim, _bm3_TestBloomFilteredLucenePostings_0.tip, _bm4.fdt, _bm4.fdx, _bm4.fnm, _bm4.nvd, _bm4.nvm, _bm4.si, _bm4_TestBloomFilteredLucenePostings_0.blm, _bm4_TestBloomFilteredLucenePostings_0.doc, _bm4_TestBloomFilteredLucenePostings_0.tim, _bm4_TestBloomFilteredLucenePostings_0.tip, _bm5.fdt, _bm5.fdx, _bm5.fnm, _bm5.nvd, _bm5.nvm, _bm5.si, _bm5_TestBloomFilteredLucenePostings_0.blm, _bm5_TestBloomFilteredLucenePostings_0.doc, _bm5_TestBloomFilteredLucenePostings_0.tim, _bm5_TestBloomFilteredLucenePostings_0.tip, _bm6.fdt, _bm6.fdx, _bm6.fnm, _bm6.nvd, _bm6.nvm, _bm6.si, _bm6_TestBloomFilteredLucenePostings_0.blm, _bm6_TestBloomFilteredLucenePostings_0.doc, _bm6_TestBloomFilteredLucenePostings_0.tim, _bm6_TestBloomFilteredLucenePostings_0.tip, _bm7.fdt, _bm7.fdx, _bm7.fnm, _bm7.nvd, _bm7.nvm, _bm7.si, _bm7_TestBloomFilteredLucenePostings_0.blm, _bm7_TestBloomFilteredLucenePostings_0.doc, _bm7_TestBloomFilteredLucenePostings_0.tim, _bm7_TestBloomFilteredLucenePostings_0.tip, _bm8.fdt, _bm8.fdx, _bm8.fnm, _bm8.nvd, _bm8.nvm, _bm8.si, _bm8_TestBloomFilteredLucenePostings_0.blm, _bm8_TestBloomFilteredLucenePostings_0.doc, _bm8_TestBloomFilteredLucenePostings_0.tim, _bm8_TestBloomFilteredLucenePostings_0.tip, _bm9.fdt, _bm9.fdx, _bm9.fnm, _bm9.nvd, _bm9.nvm, _bm9.si, _bm9_TestBloomFilteredLucenePostings_0.blm, _bm9_TestBloomFilteredLucenePostings_0.doc, _bm9_TestBloomFilteredLucenePostings_0.tim, _bm9_TestBloomFilteredLucenePostings_0.tip, _bma.fdt, _bma.fdx, _bma.fnm, _bma.nvd, _bma.nvm, _bma.si, _bma_TestBloomFilteredLucenePostings_0.blm, _bma_TestBloomFilteredLucenePostings_0.doc, _bma_TestBloomFilteredLucenePostings_0.tim, _bma_TestBloomFilteredLucenePostings_0.tip, _bmb.fdt, _bmb.fdx, _bmb.fnm, _bmb.nvd, _bmb.nvm, _bmb.si, _bmb_TestBloomFilteredLucenePostings_0.blm, _bmb_TestBloomFilteredLucenePostings_0.doc, _bmb_TestBloomFilteredLucenePostings_0.tim, _bmb_TestBloomFilteredLucenePostings_0.tip, _bmc.fdt, _bmc.fdx, _bmc.fnm, _bmc.nvd, _bmc.nvm, _bmc.si, _bmc_TestBloomFilteredLucenePostings_0.blm, _bmc_TestBloomFilteredLucenePostings_0.doc, _bmc_TestBloomFilteredLucenePostings_0.tim, _bmc_TestBloomFilteredLucenePostings_0.tip, _bmd.fdt, _bmd.fdx, _bmd.fnm, _bmd.nvd, _bmd.nvm, _bmd.si, _bmd_TestBloomFilteredLucenePostings_0.blm, _bmd_TestBloomFilteredLucenePostings_0.doc, _bmd_TestBloomFilteredLucenePostings_0.tim, _bmd_TestBloomFilteredLucenePostings_0.tip, _bme.fdt, _bme.fdx, _bme.fnm, _bme.nvd, _bme.nvm, _bme.si, _bme_TestBloomFilteredLucenePostings_0.blm, _bme_TestBloomFilteredLucenePostings_0.doc, _bme_TestBloomFilteredLucenePostings_0.tim, _bme_TestBloomFilteredLucenePostings_0.tip, _bmf.fdt, _bmf.fdx, _bmf.fnm, _bmf.nvd, _bmf.nvm, _bmf.si, _bmf_TestBloomFilteredLucenePostings_0.blm, _bmf_TestBloomFilteredLucenePostings_0.doc, _bmf_TestBloomFilteredLucenePostings_0.tim, _bmf_TestBloomFilteredLucenePostings_0.tip, _bmg.fdt, _bmg.fdx, _bmg.fnm, _bmg.nvd, _bmg.nvm, _bmg.si, _bmg_TestBloomFilteredLucenePostings_0.blm, _bmg_TestBloomFilteredLucenePostings_0.doc, _bmg_TestBloomFilteredLucenePostings_0.tim, _bmg_TestBloomFilteredLucenePostings_0.tip, _bmh.fdt, _bmh.fdx, _bmh.fnm, _bmh.nvd, _bmh.nvm, _bmh.si, _bmh_TestBloomFilteredLucenePostings_0.blm, _bmh_TestBloomFilteredLucenePostings_0.doc, _bmh_TestBloomFilteredLucenePostings_0.tim, _bmh_TestBloomFilteredLucenePostings_0.tip, _bmi.fdt, _bmi.fdx, _bmi.fnm, _bmi.nvd, _bmi.nvm, _bmi.si, _bmi_TestBloomFilteredLucenePostings_0.blm, _bmi_TestBloomFilteredLucenePostings_0.doc, _bmi_TestBloomFilteredLucenePostings_0.tim, _bmi_TestBloomFilteredLucenePostings_0.tip, _bmj.fdt, _bmj.fdx, _bmj.fnm, _bmj.nvd, _bmj.nvm, _bmj.si, _bmj_TestBloomFilteredLucenePostings_0.blm, _bmj_TestBloomFilteredLucenePostings_0.doc, _bmj_TestBloomFilteredLucenePostings_0.tim, _bmj_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_2]}]> but was:<[{indexVersion=1478134222177,generation=2,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bm1.fdt, _bm1.fdx, _bm1.fnm, _bm1.nvd, _bm1.nvm, _bm1.si, _bm1_TestBloomFilteredLucenePostings_0.blm, _bm1_TestBloomFilteredLucenePostings_0.doc, _bm1_TestBloomFilteredLucenePostings_0.tim, _bm1_TestBloomFilteredLucenePostings_0.tip, _bm2.fdt, _bm2.fdx, _bm2.fnm, _bm2.nvd, _bm2.nvm, _bm2.si, _bm2_TestBloomFilteredLucenePostings_0.blm, _bm2_TestBloomFilteredLucenePostings_0.doc, _bm2_TestBloomFilteredLucenePostings_0.tim, _bm2_TestBloomFilteredLucenePostings_0.tip, _bm3.fdt, _bm3.fdx, _bm3.fnm, _bm3.nvd, _bm3.nvm, _bm3.si, _bm3_TestBloomFilteredLucenePostings_0.blm, _bm3_TestBloomFilteredLucenePostings_0.doc, _bm3_TestBloomFilteredLucenePostings_0.tim, _bm3_TestBloomFilteredLucenePostings_0.tip, _bm4.fdt, _bm4.fdx, _bm4.fnm, _bm4.nvd, _bm4.nvm, _bm4.si, _bm4_TestBloomFilteredLucenePostings_0.blm, _bm4_TestBloomFilteredLucenePostings_0.doc, _bm4_TestBloomFilteredLucenePostings_0.tim, _bm4_TestBloomFilteredLucenePostings_0.tip, _bm5.fdt, _bm5.fdx, _bm5.fnm, _bm5.nvd, _bm5.nvm, _bm5.si, _bm5_TestBloomFilteredLucenePostings_0.blm, _bm5_TestBloomFilteredLucenePostings_0.doc, _bm5_TestBloomFilteredLucenePostings_0.tim, _bm5_TestBloomFilteredLucenePostings_0.tip, _bm6.fdt, _bm6.fdx, _bm6.fnm, _bm6.nvd, _bm6.nvm, _bm6.si, _bm6_TestBloomFilteredLucenePostings_0.blm, _bm6_TestBloomFilteredLucenePostings_0.doc, _bm6_TestBloomFilteredLucenePostings_0.tim, _bm6_TestBloomFilteredLucenePostings_0.tip, _bm7.fdt, _bm7.fdx, _bm7.fnm, _bm7.nvd, _bm7.nvm, _bm7.si, _bm7_TestBloomFilteredLucenePostings_0.blm, _bm7_TestBloomFilteredLucenePostings_0.doc, _bm7_TestBloomFilteredLucenePostings_0.tim, _bm7_TestBloomFilteredLucenePostings_0.tip, _bm8.fdt, _bm8.fdx, _bm8.fnm, _bm8.nvd, _bm8.nvm, _bm8.si, _bm8_TestBloomFilteredLucenePostings_0.blm, _bm8_TestBloomFilteredLucenePostings_0.doc, _bm8_TestBloomFilteredLucenePostings_0.tim, _bm8_TestBloomFilteredLucenePostings_0.tip, _bm9.fdt, _bm9.fdx, _bm9.fnm, _bm9.nvd, _bm9.nvm, _bm9.si, _bm9_TestBloomFilteredLucenePostings_0.blm, _bm9_TestBloomFilteredLucenePostings_0.doc, _bm9_TestBloomFilteredLucenePostings_0.tim, _bm9_TestBloomFilteredLucenePostings_0.tip, _bma.fdt, _bma.fdx, _bma.fnm, _bma.nvd, _bma.nvm, _bma.si, _bma_TestBloomFilteredLucenePostings_0.blm, _bma_TestBloomFilteredLucenePostings_0.doc, _bma_TestBloomFilteredLucenePostings_0.tim, _bma_TestBloomFilteredLucenePostings_0.tip, _bmb.fdt, _bmb.fdx, _bmb.fnm, _bmb.nvd, _bmb.nvm, _bmb.si, _bmb_TestBloomFilteredLucenePostings_0.blm, _bmb_TestBloomFilteredLucenePostings_0.doc, _bmb_TestBloomFilteredLucenePostings_0.tim, _bmb_TestBloomFilteredLucenePostings_0.tip, _bmc.fdt, _bmc.fdx, _bmc.fnm, _bmc.nvd, _bmc.nvm, _bmc.si, _bmc_TestBloomFilteredLucenePostings_0.blm, _bmc_TestBloomFilteredLucenePostings_0.doc, _bmc_TestBloomFilteredLucenePostings_0.tim, _bmc_TestBloomFilteredLucenePostings_0.tip, _bmd.fdt, _bmd.fdx, _bmd.fnm, _bmd.nvd, _bmd.nvm, _bmd.si, _bmd_TestBloomFilteredLucenePostings_0.blm, _bmd_TestBloomFilteredLucenePostings_0.doc, _bmd_TestBloomFilteredLucenePostings_0.tim, _bmd_TestBloomFilteredLucenePostings_0.tip, _bme.fdt, _bme.fdx, _bme.fnm, _bme.nvd, _bme.nvm, _bme.si, _bme_TestBloomFilteredLucenePostings_0.blm, _bme_TestBloomFilteredLucenePostings_0.doc, _bme_TestBloomFilteredLucenePostings_0.tim, _bme_TestBloomFilteredLucenePostings_0.tip, _bmf.fdt, _bmf.fdx, _bmf.fnm, _bmf.nvd, _bmf.nvm, _bmf.si, _bmf_TestBloomFilteredLucenePostings_0.blm, _bmf_TestBloomFilteredLucenePostings_0.doc, _bmf_TestBloomFilteredLucenePostings_0.tim, _bmf_TestBloomFilteredLucenePostings_0.tip, _bmg.fdt, _bmg.fdx, _bmg.fnm, _bmg.nvd, _bmg.nvm, _bmg.si, _bmg_TestBloomFilteredLucenePostings_0.blm, _bmg_TestBloomFilteredLucenePostings_0.doc, _bmg_TestBloomFilteredLucenePostings_0.tim, _bmg_TestBloomFilteredLucenePostings_0.tip, _bmh.fdt, _bmh.fdx, _bmh.fnm, _bmh.nvd, _bmh.nvm, _bmh.si, _bmh_TestBloomFilteredLucenePostings_0.blm, _bmh_TestBloomFilteredLucenePostings_0.doc, _bmh_TestBloomFilteredLucenePostings_0.tim, _bmh_TestBloomFilteredLucenePostings_0.tip, _bmi.fdt, _bmi.fdx, _bmi.fnm, _bmi.nvd, _bmi.nvm, _bmi.si, _bmi_TestBloomFilteredLucenePostings_0.blm, _bmi_TestBloomFilteredLucenePostings_0.doc, _bmi_TestBloomFilteredLucenePostings_0.tim, _bmi_TestBloomFilteredLucenePostings_0.tip, _bmj.fdt, _bmj.fdx, _bmj.fnm, _bmj.nvd, _bmj.nvm, _bmj.si, _bmj_TestBloomFilteredLucenePostings_0.blm, _bmj_TestBloomFilteredLucenePostings_0.doc, _bmj_TestBloomFilteredLucenePostings_0.tim, _bmj_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_2]}, {indexVersion=1478134222177,generation=3,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bml.fdt, _bml.fdx, _bml.fnm, _bml.nvd, _bml.nvm, _bml.si, _bml_TestBloomFilteredLucenePostings_0.blm, _bml_TestBloomFilteredLucenePostings_0.doc, _bml_TestBloomFilteredLucenePostings_0.tim, _bml_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1478134222177,generation=2,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bm1.fdt, _bm1.fdx, _bm1.fnm, _bm1.nvd, _bm1.nvm, _bm1.si, _bm1_TestBloomFilteredLucenePostings_0.blm, _bm1_TestBloomFilteredLucenePostings_0.doc, _bm1_TestBloomFilteredLucenePostings_0.tim, _bm1_TestBloomFilteredLucenePostings_0.tip, _bm2.fdt, _bm2.fdx, _bm2.fnm, _bm2.nvd, _bm2.nvm, _bm2.si, _bm2_TestBloomFilteredLucenePostings_0.blm, _bm2_TestBloomFilteredLucenePostings_0.doc, _bm2_TestBloomFilteredLucenePostings_0.tim, _bm2_TestBloomFilteredLucenePostings_0.tip, _bm3.fdt, _bm3.fdx, _bm3.fnm, _bm3.nvd, _bm3.nvm, _bm3.si, _bm3_TestBloomFilteredLucenePostings_0.blm, _bm3_TestBloomFilteredLucenePostings_0.doc, _bm3_TestBloomFilteredLucenePostings_0.tim, _bm3_TestBloomFilteredLucenePostings_0.tip, _bm4.fdt, _bm4.fdx, _bm4.fnm, _bm4.nvd, _bm4.nvm, _bm4.si, _bm4_TestBloomFilteredLucenePostings_0.blm, _bm4_TestBloomFilteredLucenePostings_0.doc, _bm4_TestBloomFilteredLucenePostings_0.tim, _bm4_TestBloomFilteredLucenePostings_0.tip, _bm5.fdt, _bm5.fdx, _bm5.fnm, _bm5.nvd, _bm5.nvm, _bm5.si, _bm5_TestBloomFilteredLucenePostings_0.blm, _bm5_TestBloomFilteredLucenePostings_0.doc, _bm5_TestBloomFilteredLucenePostings_0.tim, _bm5_TestBloomFilteredLucenePostings_0.tip, _bm6.fdt, _bm6.fdx, _bm6.fnm, _bm6.nvd, _bm6.nvm, _bm6.si, _bm6_TestBloomFilteredLucenePostings_0.blm, _bm6_TestBloomFilteredLucenePostings_0.doc, _bm6_TestBloomFilteredLucenePostings_0.tim, _bm6_TestBloomFilteredLucenePostings_0.tip, _bm7.fdt, _bm7.fdx, _bm7.fnm, _bm7.nvd, _bm7.nvm, _bm7.si, _bm7_TestBloomFilteredLucenePostings_0.blm, _bm7_TestBloomFilteredLucenePostings_0.doc, _bm7_TestBloomFilteredLucenePostings_0.tim, _bm7_TestBloomFilteredLucenePostings_0.tip, _bm8.fdt, _bm8.fdx, _bm8.fnm, _bm8.nvd, _bm8.nvm, _bm8.si, _bm8_TestBloomFilteredLucenePostings_0.blm, _bm8_TestBloomFilteredLucenePostings_0.doc, _bm8_TestBloomFilteredLucenePostings_0.tim, _bm8_TestBloomFilteredLucenePostings_0.tip, _bm9.fdt, _bm9.fdx, _bm9.fnm, _bm9.nvd, _bm9.nvm, _bm9.si, _bm9_TestBloomFilteredLucenePostings_0.blm, _bm9_TestBloomFilteredLucenePostings_0.doc, _bm9_TestBloomFilteredLucenePostings_0.tim, _bm9_TestBloomFilteredLucenePostings_0.tip, _bma.fdt, _bma.fdx, _bma.fnm, _bma.nvd, _bma.nvm, _bma.si, _bma_TestBloomFilteredLucenePostings_0.blm, _bma_TestBloomFilteredLucenePostings_0.doc, _bma_TestBloomFilteredLucenePostings_0.tim, _bma_TestBloomFilteredLucenePostings_0.tip, _bmb.fdt, _bmb.fdx, _bmb.fnm, _bmb.nvd, _bmb.nvm, _bmb.si, _bmb_TestBloomFilteredLucenePostings_0.blm, _bmb_TestBloomFilteredLucenePostings_0.doc, _bmb_TestBloomFilteredLucenePostings_0.tim, _bmb_TestBloomFilteredLucenePostings_0.tip, _bmc.fdt, _bmc.fdx, _bmc.fnm, _bmc.nvd, _bmc.nvm, _bmc.si, _bmc_TestBloomFilteredLucenePostings_0.blm, _bmc_TestBloomFilteredLucenePostings_0.doc, _bmc_TestBloomFilteredLucenePostings_0.tim, _bmc_TestBloomFilteredLucenePostings_0.tip, _bmd.fdt, _bmd.fdx, _bmd.fnm, _bmd.nvd, _bmd.nvm, _bmd.si, _bmd_TestBloomFilteredLucenePostings_0.blm, _bmd_TestBloomFilteredLucenePostings_0.doc, _bmd_TestBloomFilteredLucenePostings_0.tim, _bmd_TestBloomFilteredLucenePostings_0.tip, _bme.fdt, _bme.fdx, _bme.fnm, _bme.nvd, _bme.nvm, _bme.si, _bme_TestBloomFilteredLucenePostings_0.blm, _bme_TestBloomFilteredLucenePostings_0.doc, _bme_TestBloomFilteredLucenePostings_0.tim, _bme_TestBloomFilteredLucenePostings_0.tip, _bmf.fdt, _bmf.fdx, _bmf.fnm, _bmf.nvd, _bmf.nvm, _bmf.si, _bmf_TestBloomFilteredLucenePostings_0.blm, _bmf_TestBloomFilteredLucenePostings_0.doc, _bmf_TestBloomFilteredLucenePostings_0.tim, _bmf_TestBloomFilteredLucenePostings_0.tip, _bmg.fdt, _bmg.fdx, _bmg.fnm, _bmg.nvd, _bmg.nvm, _bmg.si, _bmg_TestBloomFilteredLucenePostings_0.blm, _bmg_TestBloomFilteredLucenePostings_0.doc, _bmg_TestBloomFilteredLucenePostings_0.tim, _bmg_TestBloomFilteredLucenePostings_0.tip, _bmh.fdt, _bmh.fdx, _bmh.fnm, _bmh.nvd, _bmh.nvm, _bmh.si, _bmh_TestBloomFilteredLucenePostings_0.blm, _bmh_TestBloomFilteredLucenePostings_0.doc, _bmh_TestBloomFilteredLucenePostings_0.tim, _bmh_TestBloomFilteredLucenePostings_0.tip, _bmi.fdt, _bmi.fdx, _bmi.fnm, _bmi.nvd, _bmi.nvm, _bmi.si, _bmi_TestBloomFilteredLucenePostings_0.blm, _bmi_TestBloomFilteredLucenePostings_0.doc, _bmi_TestBloomFilteredLucenePostings_0.tim, _bmi_TestBloomFilteredLucenePostings_0.tip, _bmj.fdt, _bmj.fdx, _bmj.fnm, _bmj.nvd, _bmj.nvm, _bmj.si, _bmj_TestBloomFilteredLucenePostings_0.blm, _bmj_TestBloomFilteredLucenePostings_0.doc, _bmj_TestBloomFilteredLucenePostings_0.tim, _bmj_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_2]}]> but was:<[{indexVersion=1478134222177,generation=2,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bm1.fdt, _bm1.fdx, _bm1.fnm, _bm1.nvd, _bm1.nvm, _bm1.si, _bm1_TestBloomFilteredLucenePostings_0.blm, _bm1_TestBloomFilteredLucenePostings_0.doc, _bm1_TestBloomFilteredLucenePostings_0.tim, _bm1_TestBloomFilteredLucenePostings_0.tip, _bm2.fdt, _bm2.fdx, _bm2.fnm, _bm2.nvd, _bm2.nvm, _bm2.si, _bm2_TestBloomFilteredLucenePostings_0.blm, _bm2_TestBloomFilteredLucenePostings_0.doc, _bm2_TestBloomFilteredLucenePostings_0.tim, _bm2_TestBloomFilteredLucenePostings_0.tip, _bm3.fdt, _bm3.fdx, _bm3.fnm, _bm3.nvd, _bm3.nvm, _bm3.si, _bm3_TestBloomFilteredLucenePostings_0.blm, _bm3_TestBloomFilteredLucenePostings_0.doc, _bm3_TestBloomFilteredLucenePostings_0.tim, _bm3_TestBloomFilteredLucenePostings_0.tip, _bm4.fdt, _bm4.fdx, _bm4.fnm, _bm4.nvd, _bm4.nvm, _bm4.si, _bm4_TestBloomFilteredLucenePostings_0.blm, _bm4_TestBloomFilteredLucenePostings_0.doc, _bm4_TestBloomFilteredLucenePostings_0.tim, _bm4_TestBloomFilteredLucenePostings_0.tip, _bm5.fdt, _bm5.fdx, _bm5.fnm, _bm5.nvd, _bm5.nvm, _bm5.si, _bm5_TestBloomFilteredLucenePostings_0.blm, _bm5_TestBloomFilteredLucenePostings_0.doc, _bm5_TestBloomFilteredLucenePostings_0.tim, _bm5_TestBloomFilteredLucenePostings_0.tip, _bm6.fdt, _bm6.fdx, _bm6.fnm, _bm6.nvd, _bm6.nvm, _bm6.si, _bm6_TestBloomFilteredLucenePostings_0.blm, _bm6_TestBloomFilteredLucenePostings_0.doc, _bm6_TestBloomFilteredLucenePostings_0.tim, _bm6_TestBloomFilteredLucenePostings_0.tip, _bm7.fdt, _bm7.fdx, _bm7.fnm, _bm7.nvd, _bm7.nvm, _bm7.si, _bm7_TestBloomFilteredLucenePostings_0.blm, _bm7_TestBloomFilteredLucenePostings_0.doc, _bm7_TestBloomFilteredLucenePostings_0.tim, _bm7_TestBloomFilteredLucenePostings_0.tip, _bm8.fdt, _bm8.fdx, _bm8.fnm, _bm8.nvd, _bm8.nvm, _bm8.si, _bm8_TestBloomFilteredLucenePostings_0.blm, _bm8_TestBloomFilteredLucenePostings_0.doc, _bm8_TestBloomFilteredLucenePostings_0.tim, _bm8_TestBloomFilteredLucenePostings_0.tip, _bm9.fdt, _bm9.fdx, _bm9.fnm, _bm9.nvd, _bm9.nvm, _bm9.si, _bm9_TestBloomFilteredLucenePostings_0.blm, _bm9_TestBloomFilteredLucenePostings_0.doc, _bm9_TestBloomFilteredLucenePostings_0.tim, _bm9_TestBloomFilteredLucenePostings_0.tip, _bma.fdt, _bma.fdx, _bma.fnm, _bma.nvd, _bma.nvm, _bma.si, _bma_TestBloomFilteredLucenePostings_0.blm, _bma_TestBloomFilteredLucenePostings_0.doc, _bma_TestBloomFilteredLucenePostings_0.tim, _bma_TestBloomFilteredLucenePostings_0.tip, _bmb.fdt, _bmb.fdx, _bmb.fnm, _bmb.nvd, _bmb.nvm, _bmb.si, _bmb_TestBloomFilteredLucenePostings_0.blm, _bmb_TestBloomFilteredLucenePostings_0.doc, _bmb_TestBloomFilteredLucenePostings_0.tim, _bmb_TestBloomFilteredLucenePostings_0.tip, _bmc.fdt, _bmc.fdx, _bmc.fnm, _bmc.nvd, _bmc.nvm, _bmc.si, _bmc_TestBloomFilteredLucenePostings_0.blm, _bmc_TestBloomFilteredLucenePostings_0.doc, _bmc_TestBloomFilteredLucenePostings_0.tim, _bmc_TestBloomFilteredLucenePostings_0.tip, _bmd.fdt, _bmd.fdx, _bmd.fnm, _bmd.nvd, _bmd.nvm, _bmd.si, _bmd_TestBloomFilteredLucenePostings_0.blm, _bmd_TestBloomFilteredLucenePostings_0.doc, _bmd_TestBloomFilteredLucenePostings_0.tim, _bmd_TestBloomFilteredLucenePostings_0.tip, _bme.fdt, _bme.fdx, _bme.fnm, _bme.nvd, _bme.nvm, _bme.si, _bme_TestBloomFilteredLucenePostings_0.blm, _bme_TestBloomFilteredLucenePostings_0.doc, _bme_TestBloomFilteredLucenePostings_0.tim, _bme_TestBloomFilteredLucenePostings_0.tip, _bmf.fdt, _bmf.fdx, _bmf.fnm, _bmf.nvd, _bmf.nvm, _bmf.si, _bmf_TestBloomFilteredLucenePostings_0.blm, _bmf_TestBloomFilteredLucenePostings_0.doc, _bmf_TestBloomFilteredLucenePostings_0.tim, _bmf_TestBloomFilteredLucenePostings_0.tip, _bmg.fdt, _bmg.fdx, _bmg.fnm, _bmg.nvd, _bmg.nvm, _bmg.si, _bmg_TestBloomFilteredLucenePostings_0.blm, _bmg_TestBloomFilteredLucenePostings_0.doc, _bmg_TestBloomFilteredLucenePostings_0.tim, _bmg_TestBloomFilteredLucenePostings_0.tip, _bmh.fdt, _bmh.fdx, _bmh.fnm, _bmh.nvd, _bmh.nvm, _bmh.si, _bmh_TestBloomFilteredLucenePostings_0.blm, _bmh_TestBloomFilteredLucenePostings_0.doc, _bmh_TestBloomFilteredLucenePostings_0.tim, _bmh_TestBloomFilteredLucenePostings_0.tip, _bmi.fdt, _bmi.fdx, _bmi.fnm, _bmi.nvd, _bmi.nvm, _bmi.si, _bmi_TestBloomFilteredLucenePostings_0.blm, _bmi_TestBloomFilteredLucenePostings_0.doc, _bmi_TestBloomFilteredLucenePostings_0.tim, _bmi_TestBloomFilteredLucenePostings_0.tip, _bmj.fdt, _bmj.fdx, _bmj.fnm, _bmj.nvd, _bmj.nvm, _bmj.si, _bmj_TestBloomFilteredLucenePostings_0.blm, _bmj_TestBloomFilteredLucenePostings_0.doc, _bmj_TestBloomFilteredLucenePostings_0.tim, _bmj_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_2]}, {indexVersion=1478134222177,generation=3,filelist=[_7vz.fdt, _7vz.fdx, _7vz.fnm, _7vz.nvd, _7vz.nvm, _7vz.si, _7vz_TestBloomFilteredLucenePostings_0.blm, _7vz_TestBloomFilteredLucenePostings_0.doc, _7vz_TestBloomFilteredLucenePostings_0.tim, _7vz_TestBloomFilteredLucenePostings_0.tip, _bmk.fdt, _bmk.fdx, _bmk.fnm, _bmk.nvd, _bmk.nvm, _bmk.si, _bmk_TestBloomFilteredLucenePostings_0.blm, _bmk_TestBloomFilteredLucenePostings_0.doc, _bmk_TestBloomFilteredLucenePostings_0.tim, _bmk_TestBloomFilteredLucenePostings_0.tip, _bml.fdt, _bml.fdx, _bml.fnm, _bml.nvd, _bml.nvm, _bml.si, _bml_TestBloomFilteredLucenePostings_0.blm, _bml_TestBloomFilteredLucenePostings_0.doc, _bml_TestBloomFilteredLucenePostings_0.tim, _bml_TestBloomFilteredLucenePostings_0.tip, _bmm.fdt, _bmm.fdx, _bmm.fnm, _bmm.nvd, _bmm.nvm, _bmm.si, _bmm_TestBloomFilteredLucenePostings_0.blm, _bmm_TestBloomFilteredLucenePostings_0.doc, _bmm_TestBloomFilteredLucenePostings_0.tim, _bmm_TestBloomFilteredLucenePostings_0.tip, _bmn.fdt, _bmn.fdx, _bmn.fnm, _bmn.nvd, _bmn.nvm, _bmn.si, _bmn_TestBloomFilteredLucenePostings_0.blm, _bmn_TestBloomFilteredLucenePostings_0.doc, _bmn_TestBloomFilteredLucenePostings_0.tim, _bmn_TestBloomFilteredLucenePostings_0.tip, _bmo.fdt, _bmo.fdx, _bmo.fnm, _bmo.nvd, _bmo.nvm, _bmo.si, _bmo_TestBloomFilteredLucenePostings_0.blm, _bmo_TestBloomFilteredLucenePostings_0.doc, _bmo_TestBloomFilteredLucenePostings_0.tim, _bmo_TestBloomFilteredLucenePostings_0.tip, _bmp.fdt, _bmp.fdx, _bmp.fnm, _bmp.nvd, _bmp.nvm, _bmp.si, _bmp_TestBloomFilteredLucenePostings_0.blm, _bmp_TestBloomFilteredLucenePostings_0.doc, _bmp_TestBloomFilteredLucenePostings_0.tim, _bmp_TestBloomFilteredLucenePostings_0.tip, _bmq.fdt, _bmq.fdx, _bmq.fnm, _bmq.nvd, _bmq.nvm, _bmq.si, _bmq_TestBloomFilteredLucenePostings_0.blm, _bmq_TestBloomFilteredLucenePostings_0.doc, _bmq_TestBloomFilteredLucenePostings_0.tim, _bmq_TestBloomFilteredLucenePostings_0.tip, _bmr.fdt, _bmr.fdx, _bmr.fnm, _bmr.nvd, _bmr.nvm, _bmr.si, _bmr_TestBloomFilteredLucenePostings_0.blm, _bmr_TestBloomFilteredLucenePostings_0.doc, _bmr_TestBloomFilteredLucenePostings_0.tim, _bmr_TestBloomFilteredLucenePostings_0.tip, _bms.fdt, _bms.fdx, _bms.fnm, _bms.nvd, _bms.nvm, _bms.si, _bms_TestBloomFilteredLucenePostings_0.blm, _bms_TestBloomFilteredLucenePostings_0.doc, _bms_TestBloomFilteredLucenePostings_0.tim, _bms_TestBloomFilteredLucenePostings_0.tip, _bmt.fdt, _bmt.fdx, _bmt.fnm, _bmt.nvd, _bmt.nvm, _bmt.si, _bmt_TestBloomFilteredLucenePostings_0.blm, _bmt_TestBloomFilteredLucenePostings_0.doc, _bmt_TestBloomFilteredLucenePostings_0.tim, _bmt_TestBloomFilteredLucenePostings_0.tip, _bmu.fdt, _bmu.fdx, _bmu.fnm, _bmu.nvd, _bmu.nvm, _bmu.si, _bmu_TestBloomFilteredLucenePostings_0.blm, _bmu_TestBloomFilteredLucenePostings_0.doc, _bmu_TestBloomFilteredLucenePostings_0.tim, _bmu_TestBloomFilteredLucenePostings_0.tip, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([150B8ABDB6F96C42:30DC918DC6B16241]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([150B8ABDB6F96C42:3875C5E741DDD64E]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={ "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-bfffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"c0000000-ffffffff", "state":"active", "replicas":{}}, "shard3":{ "range":"0-3fffffff", "state":"active", "replicas":{}}, "shard4":{ "range":"40000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-bfffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"0-3fffffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([150B8ABDB6F96C42:5D7EFE09B0CA43D7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12184 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/init-core-data-001
[junit4] 2> 2100430 INFO (SUITE-TestReplicationHandler-seed#[150B8ABDB6F96C42]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 2100432 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestStressReplication
[junit4] 2> 2100432 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1
[junit4] 2> 2100436 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2100437 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@194d944b{/solr,null,AVAILABLE}
[junit4] 2> 2100439 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Started ServerConnector@779c6d4c{HTTP/1.1,[http/1.1]}{127.0.0.1:38969}
[junit4] 2> 2100439 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server Started @2104184ms
[junit4] 2> 2100439 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=38969}
[junit4] 2> 2100440 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2100448 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 2100448 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2100448 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2100448 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-03T00:37:59.489Z
[junit4] 2> 2100448 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/solr.xml
[junit4] 2> 2100461 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2100475 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/.
[junit4] 2> 2100475 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2100488 WARN (coreLoadExecutor-5392-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2100489 INFO (coreLoadExecutor-5392-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2100504 INFO (coreLoadExecutor-5392-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2100508 INFO (coreLoadExecutor-5392-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2100511 INFO (coreLoadExecutor-5392-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/./collection1
[junit4] 2> 2100511 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2100512 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6436491549365851]
[junit4] 2> 2100524 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2100524 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2100524 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5.6298828125, floorSegmentMB=0.923828125, forceMergeDeletesPctAllowed=21.25328296385036, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2100526 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ef03177[collection1] main]
[junit4] 2> 2100526 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1/conf
[junit4] 2> 2100526 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2100526 INFO (coreLoadExecutor-5392-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2100526 INFO (searcherExecutor-5393-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6ef03177[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2100977 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1
[junit4] 2> 2100980 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2100997 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@60203f36{/solr,null,AVAILABLE}
[junit4] 2> 2100998 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Started ServerConnector@615e5814{HTTP/1.1,[http/1.1]}{127.0.0.1:53752}
[junit4] 2> 2100998 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server Started @2104742ms
[junit4] 2> 2100998 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=53752}
[junit4] 2> 2100999 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2100999 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 2100999 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2100999 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2100999 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-03T00:38:00.040Z
[junit4] 2> 2100999 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/solr.xml
[junit4] 2> 2101006 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2101013 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/.
[junit4] 2> 2101013 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2101056 WARN (coreLoadExecutor-5402-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2101057 INFO (coreLoadExecutor-5402-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2101099 INFO (coreLoadExecutor-5402-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2101103 INFO (coreLoadExecutor-5402-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2101105 INFO (coreLoadExecutor-5402-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/./collection1
[junit4] 2> 2101105 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2101108 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6436491549365851]
[junit4] 2> 2101124 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2101124 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2101125 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5.6298828125, floorSegmentMB=0.923828125, forceMergeDeletesPctAllowed=21.25328296385036, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2101125 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@63fe1706[collection1] main]
[junit4] 2> 2101125 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1/conf
[junit4] 2> 2101128 INFO (coreLoadExecutor-5402-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2101128 INFO (searcherExecutor-5403-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@63fe1706[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2101132 INFO (qtp271446332-67636) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2101141 INFO (indexFetcher-5407-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2101141 INFO (indexFetcher-5407-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2101141 INFO (indexFetcher-5407-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2101540 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@615e5814{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2101540 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=154204851
[junit4] 2> 2101541 INFO (coreCloseExecutor-5408-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@6cf2704b
[junit4] 2> 2101543 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@60203f36{/solr,null,UNAVAILABLE}
[junit4] 2> 2101548 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2101557 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@276292a2{/solr,null,AVAILABLE}
[junit4] 2> 2101558 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Started ServerConnector@19b93482{HTTP/1.1,[http/1.1]}{127.0.0.1:44321}
[junit4] 2> 2101558 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server Started @2105303ms
[junit4] 2> 2101558 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=44321}
[junit4] 2> 2101558 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2101558 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 2101559 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2101559 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2101559 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-03T00:38:00.600Z
[junit4] 2> 2101559 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/solr.xml
[junit4] 2> 2101566 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2101568 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/.
[junit4] 2> 2101568 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2101594 WARN (coreLoadExecutor-5415-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2101595 INFO (coreLoadExecutor-5415-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2101617 INFO (coreLoadExecutor-5415-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2101620 INFO (coreLoadExecutor-5415-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2101638 INFO (coreLoadExecutor-5415-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/./collection1
[junit4] 2> 2101639 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2101639 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6436491549365851]
[junit4] 2> 2101647 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2101648 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2101648 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5.6298828125, floorSegmentMB=0.923828125, forceMergeDeletesPctAllowed=21.25328296385036, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2101648 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c99639d[collection1] main]
[junit4] 2> 2101648 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-002/collection1/conf
[junit4] 2> 2101649 INFO (coreLoadExecutor-5415-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2101649 INFO (searcherExecutor-5416-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4c99639d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2102086 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@779c6d4c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2102086 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=416469007
[junit4] 2> 2102086 INFO (coreCloseExecutor-5420-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5e7c8e18
[junit4] 2> 2102090 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@194d944b{/solr,null,UNAVAILABLE}
[junit4] 2> 2102092 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2102106 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@fd5f7fd{/solr,null,AVAILABLE}
[junit4] 2> 2102106 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.ServerConnector Started ServerConnector@7e70ca25{HTTP/1.1,[http/1.1]}{127.0.0.1:58928}
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.e.j.s.Server Started @2105851ms
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=58928}
[junit4] 2> 2102108 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.4.0
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2102108 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-03T00:38:01.149Z
[junit4] 2> 2102109 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/solr.xml
[junit4] 2> 2102119 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2102120 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/.
[junit4] 2> 2102121 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[150B8ABDB6F96C42]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2102149 WARN (coreLoadExecutor-5427-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2102150 INFO (coreLoadExecutor-5427-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 2102187 INFO (coreLoadExecutor-5427-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2102189 INFO (coreLoadExecutor-5427-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2102190 INFO (coreLoadExecutor-5427-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/./collection1
[junit4] 2> 2102190 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2102191 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=28, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6436491549365851]
[junit4] 2> 2102213 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2102213 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2102216 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5.6298828125, floorSegmentMB=0.923828125, forceMergeDeletesPctAllowed=21.25328296385036, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2102218 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@32b68efd[collection1] main]
[junit4] 2> 2102218 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_150B8ABDB6F96C42-001/solr-instance-001/collection1/conf
[junit4] 2> 2102218 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 2102218 INFO (coreLoadExecutor-5427-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2102221 INFO (searcherExecutor-5428-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@32b68efd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2102623 INFO (qtp666670479-67690) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 2102624 INFO (qtp666670479-67690) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 2102625 INFO (qtp2141037431-67672) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 2102625 INFO (qtp2141037431-67672) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 2102627 INFO (qtp2141037431-67673) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2102627 INFO (qtp2141037431-67673) [ x:collection1] o.a.s.u.SolrIndexWriter
[...truncated too long message...]
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 4334760 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[150B8ABDB6F96C42]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40242 40242
[junit4] 2> 4334761 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[150B8ABDB6F96C42]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_150B8ABDB6F96C42-001
[junit4] 2> Nov 03, 2016 1:15:13 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1240, maxMBSortInHeap=5.444203217586658, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=ar-LB, timezone=NET
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=303948944,total=514326528
[junit4] 2> NOTE: All tests run in this JVM: [TriLevelCompositeIdRoutingTest, TestDefaultStatsCache, BigEndianAscendingWordSerializerTest, DateRangeFieldTest, TestClassicSimilarityFactory, ClusterStateTest, RecoveryAfterSoftCommitTest, TestBinaryResponseWriter, TestSolrConfigHandlerConcurrent, TestSchemaManager, TestStressRecovery, UniqFieldsUpdateProcessorFactoryTest, TestFaceting, TolerantUpdateProcessorTest, BlockCacheTest, TestQuerySenderNoQuery, CreateCollectionCleanupTest, TestFileDictionaryLookup, TestPHPSerializedResponseWriter, TestSolrCloudWithKerberosAlt, SSLMigrationTest, BadCopyFieldTest, TestSolrCloudSnapshots, TestHdfsUpdateLog, TestInitQParser, TestMiniSolrCloudClusterSSL, LeaderElectionContextKeyTest, DistributedFacetPivotLongTailTest, TestRecoveryHdfs, TestTestInjection, ConcurrentDeleteAndCreateCollectionTest, DistributedFacetPivotWhiteBoxTest, TestConfigSetImmutable, TestFunctionQuery, HdfsThreadLeakTest, TestSolr4Spatial, MergeStrategyTest, NotRequiredUniqueKeyTest, TestSolrDeletionPolicy2, HdfsWriteToMultipleCollectionsTest, ExternalFileFieldSortTest, TestDynamicFieldResource, TestSubQueryTransformerDistrib, TestTolerantUpdateProcessorCloud, TestDistributedSearch, TestRawTransformer, HighlighterConfigTest, DateFieldTest, TestUniqueKeyFieldResource, SolrRequestParserTest, TestJoin, SyncSliceTest, TestSolrCoreProperties, TestLeaderElectionZkExpiry, LeaderElectionIntegrationTest, TestRangeQuery, TestBlobHandler, CacheHeaderTest, AnalysisErrorHandlingTest, DistributedVersionInfoTest, SegmentsInfoRequestHandlerTest, TestFieldCacheSort, UUIDFieldTest, PeerSyncWithIndexFingerprintCachingTest, FastVectorHighlighterTest, TestFieldCacheWithThreads, SolrCLIZkUtilsTest, DefaultValueUpdateProcessorTest, SmileWriterTest, TestFieldTypeCollectionResource, TestValueSourceCache, NumericFieldsTest, AlternateDirectoryTest, TestDeleteCollectionOnDownNodes, ZkNodePropsTest, TestExclusionRuleCollectionAccess, TestSQLHandler, TestIBSimilarityFactory, HdfsRecoveryZkTest, RegexBoostProcessorTest, DeleteLastCustomShardedReplicaTest, TestRandomDVFaceting, PKIAuthenticationIntegrationTest, BasicDistributedZkTest, TestSchemaResource, TestPerFieldSimilarity, TestComplexPhraseQParserPlugin, TestFilteredDocIdSet, TestEmbeddedSolrServerConstructors, TestDocumentBuilder, AtomicUpdatesTest, TestDocSet, TestShortCircuitedRequests, SynonymTokenizerTest, SmileWriterTest, FullHLLTest, TestCorePropertiesReload, DocExpirationUpdateProcessorFactoryTest, JavabinLoaderTest, QueryResultKeyTest, TestBadConfig, DistributedSuggestComponentTest, ConjunctionSolrSpellCheckerTest, TestAtomicUpdateErrorCases, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, TestHashPartitioner, ZkControllerTest, TestRealTimeGet, TestStressVersions, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, QueryElevationComponentTest, PeerSyncTest, TestLazyCores, DirectUpdateHandlerTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, TestStressLucene, TestTrie, TestCSVLoader, PolyFieldTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, DebugComponentTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, TermsComponentTest, DocumentBuilderTest, TermVectorComponentTest, RequiredFieldsTest, LoggingHandlerTest, TestJmxIntegration, MBeansHandlerTest, BinaryUpdateRequestHandlerTest, TestQuerySenderListener, CopyFieldTest, SolrIndexConfigTest, TestSearchPerf, MinimalSchemaTest, OutputWriterTest, SpellPossibilityIteratorTest, TestCodecSupport, TestLMJelinekMercerSimilarityFactory, TestJmxMonitoredMap, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, AsyncCallRequestStatusResponseTest, CdcrVersionReplicationTest, CollectionsAPISolrJTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, HttpPartitionTest, MultiThreadedOCPTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerModifyCollectionTest, OverseerStatusTest, OverseerTaskQueueTest, ReplaceNodeTest, ReplicationFactorTest, RestartWhileUpdatingTest, TestCloudInspectUtil, TestCloudPivotFacet, TestLeaderElectionWithEmptyReplica, TestLockTree, TestOnReconnectListenerSupport, TestRandomRequestDistribution, TestRebalanceLeaders, TestReplicaProperties, TestRequestForwarding, TestRequestStatusCollectionAPI, TestSSLRandomization, TestSizeLimitedDistributedMap, TestTolerantUpdateProcessorRandomCloud, HdfsBasicDistributedZkTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [513/648 (4!)] on J0 in 172.98s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 60809 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 191 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/191/
4 tests failed.
FAILED: org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig
Error Message:
The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
at __randomizedtesting.SeedInfo.seed([F1F1B13441B49802:65E781FBC3D7229]:0)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:304)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:280)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:220)
at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:63)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:302)
... 42 more
FAILED: org.apache.solr.search.mlt.SimpleMLTQParserTest.doTest
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([F1F1B13441B49802:56B509902C0F8BBB]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:812)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:779)
at org.apache.solr.search.mlt.SimpleMLTQParserTest.doTest(SimpleMLTQParserTest.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='13']
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">18</int></lst><result name="response" numFound="16" start="0"><doc><int name="id">30</int><arr name="range_facet_l"><long>30</long></arr><arr name="range_facet_l_dv"><long>30</long></arr><arr name="range_facet_i_dv"><int>30</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str><str>red green yellow</str></arr><arr name="lowerfilt1"><str>red green yellow</str></arr><long name="_version_">1549840936027029505</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.204Z</date></doc><doc><int name="id">31</int><arr name="range_facet_l"><long>31</long></arr><arr name="range_facet_l_dv"><long>31</long></arr><arr name="range_facet_i_dv"><int>31</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str><str>green blue yellow</str></arr><arr name="lowerfilt1"><str>green blue yellow</str></arr><long name="_version_">1549840936028078080</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.205Z</date></doc><doc><int name="id">13</int><arr name="range_facet_l"><long>13</long></arr><arr name="range_facet_l_dv"><long>13</long></arr><arr name="range_facet_i_dv"><int>13</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936008155136</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.185Z</date></doc><doc><int name="id">14</int><arr name="range_facet_l"><long>14</long></arr><arr name="range_facet_l_dv"><long>14</long></arr><arr name="range_facet_i_dv"><int>14</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252288</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">15</int><arr name="range_facet_l"><long>15</long></arr><arr name="range_facet_l_dv"><long>15</long></arr><arr name="range_facet_i_dv"><int>15</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252289</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">16</int><arr name="range_facet_l"><long>16</long></arr><arr name="range_facet_l_dv"><long>16</long></arr><arr name="range_facet_i_dv"><int>16</int></arr><arr name="lowerfilt"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936011300864</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.189Z</date></doc><doc><int name="id">18</int><arr name="range_facet_l"><long>18</long></arr><arr name="range_facet_l_dv"><long>18</long></arr><arr name="range_facet_i_dv"><int>18</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446592</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">19</int><arr name="range_facet_l"><long>19</long></arr><arr name="range_facet_l_dv"><long>19</long></arr><arr name="range_facet_i_dv"><int>19</int></arr><arr name="lowerfilt"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446593</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">20</int><arr name="range_facet_l"><long>20</long></arr><arr name="range_facet_l_dv"><long>20</long></arr><arr name="range_facet_i_dv"><int>20</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592320</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.194Z</date></doc><doc><int name="id">21</int><arr name="range_facet_l"><long>21</long></arr><arr name="range_facet_l_dv"><long>21</long></arr><arr name="range_facet_i_dv"><int>21</int></arr><arr name="lowerfilt"><str>The court red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The court red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592321</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.195Z</date></doc></result>
</response>
request was:q={!mlt+qf%3Dlowerfilt}17
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:805)
... 41 more
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([F1F1B13441B49802:DC8FFE6EB690220E]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={ "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-b332ffff", "state":"active", "replicas":{}}, "shard2":{ "range":"b3330000-e665ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"e6660000-1998ffff", "state":"active", "replicas":{}}, "shard4":{ "range":"19990000-4ccbffff", "state":"active", "replicas":{}}, "shard5":{ "range":"4ccc0000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-b332ffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"b3330000-e665ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"e6660000-1998ffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"19990000-4ccbffff",
"state":"active",
"replicas":{}},
"shard5":{
"range":"4ccc0000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([F1F1B13441B49802:B984C5804787B797]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11914 lines...]
[junit4] Suite: org.apache.solr.search.mlt.SimpleMLTQParserTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.search.mlt.SimpleMLTQParserTest_F1F1B13441B49802-001/init-core-data-001
[junit4] 2> 1758482 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1758483 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1758484 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1758512 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1758538 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 1758640 WARN (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.s.IndexSchema [null] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1758642 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1758653 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
[junit4] 2> 1758655 INFO (coreLoadExecutor-5900-thread-1) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1758692 INFO (coreLoadExecutor-5900-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 1758718 INFO (coreLoadExecutor-5900-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1758816 WARN (coreLoadExecutor-5900-thread-1) [ ] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1758820 INFO (coreLoadExecutor-5900-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1758827 INFO (coreLoadExecutor-5900-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1
[junit4] 2> 1758827 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.search.mlt.SimpleMLTQParserTest_F1F1B13441B49802-001/init-core-data-001/]
[junit4] 2> 1758827 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@47fc9d0c
[junit4] 2> 1758829 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=8, maxMergedSegmentMB=0.94921875, floorSegmentMB=0.734375, forceMergeDeletesPctAllowed=28.488250383094904, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1758844 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1758844 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1758845 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1758845 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1758845 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=3, maxMergedSegmentMB=31.1640625, floorSegmentMB=1.5107421875, forceMergeDeletesPctAllowed=23.542472898654527, segmentsPerTier=8.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1758846 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a984aac[collection1] main]
[junit4] 2> 1758846 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 1758853 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 1758882 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1758883 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 1758883 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 1758884 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 1758884 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 1758884 INFO (coreLoadExecutor-5900-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549840935981940736
[junit4] 2> 1758884 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 1758884 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 1758885 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 1758885 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 1758885 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 1758886 INFO (searcherExecutor-5901-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a984aac[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1758887 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 1758890 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTest
[junit4] 2> 1758893 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[1 (1549840935988232192)]} 0 3
[junit4] 2> 1758895 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[2 (1549840935993475072)]} 0 0
[junit4] 2> 1758899 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[3 (1549840935994523648)]} 0 3
[junit4] 2> 1758900 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[4 (1549840935997669376)]} 0 0
[junit4] 2> 1758900 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[5 (1549840935998717952)]} 0 0
[junit4] 2> 1758902 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[6 (1549840935998717953)]} 0 2
[junit4] 2> 1758904 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[7 (1549840936001863680)]} 0 0
[junit4] 2> 1758904 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[8 (1549840936002912256)]} 0 0
[junit4] 2> 1758907 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[9 (1549840936003960832)]} 0 2
[junit4] 2> 1758908 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[10 (1549840936006057984)]} 0 0
[junit4] 2> 1758908 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[12 (1549840936007106560)]} 0 0
[junit4] 2> 1758910 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[13 (1549840936008155136)]} 0 1
[junit4] 2> 1758911 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[14 (1549840936010252288)]} 0 0
[junit4] 2> 1758912 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[15 (1549840936010252289)]} 0 0
[junit4] 2> 1758913 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[16 (1549840936011300864)]} 0 1
[junit4] 2> 1758914 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[17 (1549840936013398016)]} 0 0
[junit4] 2> 1758915 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[18 (1549840936014446592)]} 0 0
[junit4] 2> 1758917 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[19 (1549840936014446593)]} 0 2
[junit4] 2> 1758918 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[20 (1549840936017592320)]} 0 0
[junit4] 2> 1758918 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[21 (1549840936017592321)]} 0 0
[junit4] 2> 1758920 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[22 (1549840936018640896)]} 0 1
[junit4] 2> 1758921 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[23 (1549840936020738048)]} 0 0
[junit4] 2> 1758922 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[24 (1549840936020738049)]} 0 0
[junit4] 2> 1758924 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[25 (1549840936021786624)]} 0 1
[junit4] 2> 1758924 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[26 (1549840936023883776)]} 0 0
[junit4] 2> 1758925 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[27 (1549840936024932352)]} 0 0
[junit4] 2> 1758926 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[28 (1549840936024932353)]} 0 1
[junit4] 2> 1758927 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[29 (1549840936027029504)]} 0 0
[junit4] 2> 1758928 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[30 (1549840936027029505)]} 0 0
[junit4] 2> 1758931 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[31 (1549840936028078080)]} 0 3
[junit4] 2> 1758939 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{add=[32 (1549840936031223808)]} 0 7
[junit4] 2> 1758939 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1758939 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@637c3375
[junit4] 2> 1758946 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2c79fae[collection1] main]
[junit4] 2> 1758946 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1758947 INFO (searcherExecutor-5901-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2c79fae[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_h(6.4.0):C9) Uninverting(_e(6.4.0):C12) Uninverting(_g(6.4.0):C9) Uninverting(_f(6.4.0):c1)))}
[junit4] 2> 1758948 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 8
[junit4] 2> 1758967 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.c.S.Request [collection1] webapp=null path=null params={q={!mlt+qf%3Dlowerfilt}17} hits=16 status=0 QTime=18
[junit4] 2> 1758968 ERROR (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='13']
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">18</int></lst><result name="response" numFound="16" start="0"><doc><int name="id">30</int><arr name="range_facet_l"><long>30</long></arr><arr name="range_facet_l_dv"><long>30</long></arr><arr name="range_facet_i_dv"><int>30</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str><str>red green yellow</str></arr><arr name="lowerfilt1"><str>red green yellow</str></arr><long name="_version_">1549840936027029505</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.204Z</date></doc><doc><int name="id">31</int><arr name="range_facet_l"><long>31</long></arr><arr name="range_facet_l_dv"><long>31</long></arr><arr name="range_facet_i_dv"><int>31</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str><str>green blue yellow</str></arr><arr name="lowerfilt1"><str>green blue yellow</str></arr><long name="_version_">1549840936028078080</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.205Z</date></doc><doc><int name="id">13</int><arr name="range_facet_l"><long>13</long></arr><arr name="range_facet_l_dv"><long>13</long></arr><arr name="range_facet_i_dv"><int>13</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936008155136</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.185Z</date></doc><doc><int name="id">14</int><arr name="range_facet_l"><long>14</long></arr><arr name="range_facet_l_dv"><long>14</long></arr><arr name="range_facet_i_dv"><int>14</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252288</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">15</int><arr name="range_facet_l"><long>15</long></arr><arr name="range_facet_l_dv"><long>15</long></arr><arr name="range_facet_i_dv"><int>15</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252289</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">16</int><arr name="range_facet_l"><long>16</long></arr><arr name="range_facet_l_dv"><long>16</long></arr><arr name="range_facet_i_dv"><int>16</int></arr><arr name="lowerfilt"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936011300864</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.189Z</date></doc><doc><int name="id">18</int><arr name="range_facet_l"><long>18</long></arr><arr name="range_facet_l_dv"><long>18</long></arr><arr name="range_facet_i_dv"><int>18</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446592</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">19</int><arr name="range_facet_l"><long>19</long></arr><arr name="range_facet_l_dv"><long>19</long></arr><arr name="range_facet_i_dv"><int>19</int></arr><arr name="lowerfilt"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446593</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">20</int><arr name="range_facet_l"><long>20</long></arr><arr name="range_facet_l_dv"><long>20</long></arr><arr name="range_facet_i_dv"><int>20</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592320</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.194Z</date></doc><doc><int name="id">21</int><arr name="range_facet_l"><long>21</long></arr><arr name="range_facet_l_dv"><long>21</long></arr><arr name="range_facet_i_dv"><int>21</int></arr><arr name="lowerfilt"><str>The court red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The court red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592321</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.195Z</date></doc></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q={!mlt+qf%3Dlowerfilt}17
[junit4] 2> 1758970 ERROR (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q={!mlt+qf%3Dlowerfilt}17:java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='13']
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">18</int></lst><result name="response" numFound="16" start="0"><doc><int name="id">30</int><arr name="range_facet_l"><long>30</long></arr><arr name="range_facet_l_dv"><long>30</long></arr><arr name="range_facet_i_dv"><int>30</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str><str>red green yellow</str></arr><arr name="lowerfilt1"><str>red green yellow</str></arr><long name="_version_">1549840936027029505</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.204Z</date></doc><doc><int name="id">31</int><arr name="range_facet_l"><long>31</long></arr><arr name="range_facet_l_dv"><long>31</long></arr><arr name="range_facet_i_dv"><int>31</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str><str>green blue yellow</str></arr><arr name="lowerfilt1"><str>green blue yellow</str></arr><long name="_version_">1549840936028078080</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.205Z</date></doc><doc><int name="id">13</int><arr name="range_facet_l"><long>13</long></arr><arr name="range_facet_l_dv"><long>13</long></arr><arr name="range_facet_i_dv"><int>13</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936008155136</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.185Z</date></doc><doc><int name="id">14</int><arr name="range_facet_l"><long>14</long></arr><arr name="range_facet_l_dv"><long>14</long></arr><arr name="range_facet_i_dv"><int>14</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252288</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">15</int><arr name="range_facet_l"><long>15</long></arr><arr name="range_facet_l_dv"><long>15</long></arr><arr name="range_facet_i_dv"><int>15</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252289</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">16</int><arr name="range_facet_l"><long>16</long></arr><arr name="range_facet_l_dv"><long>16</long></arr><arr name="range_facet_i_dv"><int>16</int></arr><arr name="lowerfilt"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936011300864</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.189Z</date></doc><doc><int name="id">18</int><arr name="range_facet_l"><long>18</long></arr><arr name="range_facet_l_dv"><long>18</long></arr><arr name="range_facet_i_dv"><int>18</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446592</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">19</int><arr name="range_facet_l"><long>19</long></arr><arr name="range_facet_l_dv"><long>19</long></arr><arr name="range_facet_i_dv"><int>19</int></arr><arr name="lowerfilt"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446593</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">20</int><arr name="range_facet_l"><long>20</long></arr><arr name="range_facet_l_dv"><long>20</long></arr><arr name="range_facet_i_dv"><int>20</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592320</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.194Z</date></doc><doc><int name="id">21</int><arr name="range_facet_l"><long>21</long></arr><arr name="range_facet_l_dv"><long>21</long></arr><arr name="range_facet_i_dv"><int>21</int></arr><arr name="lowerfilt"><str>The court red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The court red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592321</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.195Z</date></doc></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q={!mlt+qf%3Dlowerfilt}17
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:805)
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:779)
[junit4] 2> at org.apache.solr.search.mlt.SimpleMLTQParserTest.doTest(SimpleMLTQParserTest.java:82)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1758971 INFO (TEST-SimpleMLTQParserTest.doTest-seed#[F1F1B13441B49802]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTest
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SimpleMLTQParserTest -Dtests.method=doTest -Dtests.seed=F1F1B13441B49802 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=zh-CN -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.09s J2 | SimpleMLTQParserTest.doTest <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4] > at __randomizedtesting.SeedInfo.seed([F1F1B13441B49802:56B509902C0F8BBB]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:812)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:779)
[junit4] > at org.apache.solr.search.mlt.SimpleMLTQParserTest.doTest(SimpleMLTQParserTest.java:82)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result/doc[1]/int[@name='id'][.='13']
[junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] > <response>
[junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">18</int></lst><result name="response" numFound="16" start="0"><doc><int name="id">30</int><arr name="range_facet_l"><long>30</long></arr><arr name="range_facet_l_dv"><long>30</long></arr><arr name="range_facet_i_dv"><int>30</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str><str>red green yellow</str></arr><arr name="lowerfilt1"><str>red green yellow</str></arr><long name="_version_">1549840936027029505</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.204Z</date></doc><doc><int name="id">31</int><arr name="range_facet_l"><long>31</long></arr><arr name="range_facet_l_dv"><long>31</long></arr><arr name="range_facet_i_dv"><int>31</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str><str>green blue yellow</str></arr><arr name="lowerfilt1"><str>green blue yellow</str></arr><long name="_version_">1549840936028078080</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.205Z</date></doc><doc><int name="id">13</int><arr name="range_facet_l"><long>13</long></arr><arr name="range_facet_l_dv"><long>13</long></arr><arr name="range_facet_i_dv"><int>13</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936008155136</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.185Z</date></doc><doc><int name="id">14</int><arr name="range_facet_l"><long>14</long></arr><arr name="range_facet_l_dv"><long>14</long></arr><arr name="range_facet_i_dv"><int>14</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252288</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">15</int><arr name="range_facet_l"><long>15</long></arr><arr name="range_facet_l_dv"><long>15</long></arr><arr name="range_facet_i_dv"><int>15</int></arr><arr name="lowerfilt"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The fat red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936010252289</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.188Z</date></doc><doc><int name="id">16</int><arr name="range_facet_l"><long>16</long></arr><arr name="range_facet_l_dv"><long>16</long></arr><arr name="range_facet_i_dv"><int>16</int></arr><arr name="lowerfilt"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The slim red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936011300864</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.189Z</date></doc><doc><int name="id">18</int><arr name="range_facet_l"><long>18</long></arr><arr name="range_facet_l_dv"><long>18</long></arr><arr name="range_facet_i_dv"><int>18</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446592</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">19</int><arr name="range_facet_l"><long>19</long></arr><arr name="range_facet_l_dv"><long>19</long></arr><arr name="range_facet_i_dv"><int>19</int></arr><arr name="lowerfilt"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The hose red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936014446593</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.192Z</date></doc><doc><int name="id">20</int><arr name="range_facet_l"><long>20</long></arr><arr name="range_facet_l_dv"><long>20</long></arr><arr name="range_facet_i_dv"><int>20</int></arr><arr name="lowerfilt"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The quote red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592320</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.194Z</date></doc><doc><int name="id">21</int><arr name="range_facet_l"><long>21</long></arr><arr name="range_facet_l_dv"><long>21</long></arr><arr name="range_facet_i_dv"><int>21</int></arr><arr name="lowerfilt"><str>The court red fox jumped over the lazy brown dogs.</str></arr><arr name="lowerfilt1and2"><str>The court red fox jumped over the lazy brown dogs.</str></arr><long name="_version_">1549840936017592321</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2016-11-01T23:38:15.195Z</date></doc></result>
[junit4] > </response>
[junit4] > request was:q={!mlt+qf%3Dlowerfilt}17
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:805)
[junit4] > ... 41 more
[junit4] 2> 1758978 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 1758978 INFO (SUITE-SimpleMLTQParserTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=40907112
[junit4] 2> 1758979 INFO (coreCloseExecutor-5906-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@f68634e
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.search.mlt.SimpleMLTQParserTest_F1F1B13441B49802-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {range_facet_l_dv=FST50, _version_=PostingsFormat(name=Direct), lowerfilt1=BlockTreeOrds(blocksize=128), lowerfilt1and2=PostingsFormat(name=MockRandom), multiDefault=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=Direct), lowerfilt=PostingsFormat(name=MockRandom), id=FST50, range_facet_i_dv=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=MockRandom), timestamp=PostingsFormat(name=MockRandom)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Lucene54), lowerfilt1=DocValuesFormat(name=Memory), lowerfilt1and2=DocValuesFormat(name=Lucene54), multiDefault=DocValuesFormat(name=Lucene54), intDefault=DocValuesFormat(name=Lucene54), lowerfilt=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Lucene54), id=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene54), range_facet_l=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1719, maxMBSortInHeap=7.205512415785264, sim=ClassicSimilarity, locale=zh-CN, timezone=WET
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (32-bit)/cpus=4,threads=1,free=72925416,total=530579456
[junit4] 2> NOTE: All tests run in this JVM: [UnloadDistributedZkTest, HdfsChaosMonkeySafeLeaderTest, MinimalSchemaTest, TestNamedUpdateProcessors, TestReplicaProperties, DOMUtilTest, TestSort, OverriddenZkACLAndCredentialsProvidersTest, SimpleCollectionCreateDeleteTest, CdcrBootstrapTest, RequestLoggingTest, PingRequestHandlerTest, TestCustomDocTransformer, TestFieldSortValues, DeleteInactiveReplicaTest, DistributedDebugComponentTest, TestStressReorder, TestSlowCompositeReaderWrapper, TestSolrCloudWithDelegationTokens, OpenExchangeRatesOrgProviderTest, CoreAdminHandlerTest, RollingRestartTest, DataDrivenBlockJoinTest, TestInitParams, TestLFUCache, TestCollapseQParserPlugin, TestSolrCLIRunExample, UpdateParamsTest, HdfsRecoverLeaseTest, TestShardHandlerFactory, TestHighFrequencyDictionaryFactory, TestDocTermOrds, TestConfigSets, CoreSorterTest, TestAnalyzedSuggestions, DirectUpdateHandlerTest, TestCloudInspectUtil, DocValuesMultiTest, SuggesterWFSTTest, TestWriterPerf, CloneFieldUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestManagedSynonymFilterFactory, MigrateRouteKeyTest, TestDistributedMissingSort, StatelessScriptUpdateProcessorFactoryTest, TestQueryWrapperFilter, TestPKIAuthenticationPlugin, TestUtils, TestSolrQueryParser, ParsingFieldUpdateProcessorsTest, ExplicitHLLTest, TestZkChroot, OverseerTest, TestTrackingShardHandlerFactory, DistributedQueryComponentOptimizationTest, TestSolrConfigHandlerCloud, CdcrRequestHandlerTest, HdfsUnloadDistributedZkTest, DistribCursorPagingTest, BlockJoinFacetSimpleTest, OverseerRolesTest, BasicDistributedZk2Test, TestDynamicLoading, DirectUpdateHandlerOptimizeTest, TestXIncludeConfig, TestCollationField, TestCloudManagedSchema, NoCacheHeaderTest, JSONWriterTest, PeerSyncReplicationTest, LukeRequestHandlerTest, HttpPartitionTest, TestCryptoKeys, TestSolrJ, TestFieldCacheSanityChecker, ReplaceNodeTest, TestCoreDiscovery, PluginInfoTest, DistributedFacetPivotLargeTest, CollectionsAPISolrJTest, TestExactSharedStatsCache, TestSearcherReuse, TestConfigSetsAPI, TestLMJelinekMercerSimilarityFactory, XmlUpdateRequestHandlerTest, PrimitiveFieldTypeTest, BadIndexSchemaTest, SimpleMLTQParserTest]
[junit4] Completed [253/648 (1!)] on J2 in 0.54s, 1 test, 1 error <<< FAILURES!
[...truncated 829 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/init-core-data-001
[junit4] 2> 3935713 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3935713 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/tempDir-001
[junit4] 2> 3935713 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3935714 INFO (Thread-16940) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3935714 INFO (Thread-16940) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3935814 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:32828
[junit4] 2> 3935823 INFO (jetty-launcher-32956-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 3935823 INFO (jetty-launcher-32956-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 3935823 INFO (jetty-launcher-32956-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 3935824 INFO (jetty-launcher-32956-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 3935827 INFO (jetty-launcher-32956-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79293a64{/solr,null,AVAILABLE}
[junit4] 2> 3935828 INFO (jetty-launcher-32956-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15edaa92{/solr,null,AVAILABLE}
[junit4] 2> 3935828 INFO (jetty-launcher-32956-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29da0200{/solr,null,AVAILABLE}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ce25056{/solr,null,AVAILABLE}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@1dd4a234{HTTP/1.1,[http/1.1]}{127.0.0.1:47746}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@2c42ab48{HTTP/1.1,[http/1.1]}{127.0.0.1:45096}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-2) [ ] o.e.j.s.Server Started @3938875ms
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-3) [ ] o.e.j.s.Server Started @3938875ms
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47746}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45096}
[junit4] 2> 3935830 INFO (jetty-launcher-32956-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@2ff5ebe3{HTTP/1.1,[http/1.1]}{127.0.0.1:42928}
[junit4] 2> 3935829 INFO (jetty-launcher-32956-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@68e8271b{HTTP/1.1,[http/1.1]}{127.0.0.1:38150}
[junit4] 2> 3935830 ERROR (jetty-launcher-32956-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3935830 INFO (jetty-launcher-32956-thread-4) [ ] o.e.j.s.Server Started @3938876ms
[junit4] 2> 3935831 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38150}
[junit4] 2> 3935830 ERROR (jetty-launcher-32956-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3935832 ERROR (jetty-launcher-32956-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-02T00:14:31.470Z
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-02T00:14:31.470Z
[junit4] 2> 3935830 INFO (jetty-launcher-32956-thread-1) [ ] o.e.j.s.Server Started @3938876ms
[junit4] 2> 3935832 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42928}
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-02T00:14:31.470Z
[junit4] 2> 3935833 ERROR (jetty-launcher-32956-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 3935833 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3935834 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3935834 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-11-02T00:14:31.471Z
[junit4] 2> 3935835 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3935836 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3935840 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3935848 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3935848 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3935850 INFO (jetty-launcher-32956-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32828/solr
[junit4] 2> 3935851 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15822631fb80003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3935852 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3935853 INFO (jetty-launcher-32956-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32828/solr
[junit4] 2> 3935855 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3935856 INFO (jetty-launcher-32956-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32828/solr
[junit4] 2> 3935861 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3935862 INFO (jetty-launcher-32956-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:32828/solr
[junit4] 2> 3935889 INFO (jetty-launcher-32956-thread-2) [n:127.0.0.1:47746_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47746_solr
[junit4] 2> 3935890 INFO (jetty-launcher-32956-thread-4) [n:127.0.0.1:38150_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38150_solr
[junit4] 2> 3935890 INFO (jetty-launcher-32956-thread-1) [n:127.0.0.1:42928_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42928_solr
[junit4] 2> 3935890 INFO (jetty-launcher-32956-thread-2) [n:127.0.0.1:47746_solr ] o.a.s.c.Overseer Overseer (id=96865201117724679-127.0.0.1:47746_solr-n_0000000000) starting
[junit4] 2> 3935892 INFO (zkCallback-32970-thread-1-processing-n:127.0.0.1:47746_solr) [n:127.0.0.1:47746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3935892 INFO (zkCallback-32977-thread-1-processing-n:127.0.0.1:42928_solr) [n:127.0.0.1:42928_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3935897 INFO (jetty-launcher-32956-thread-3) [n:127.0.0.1:45096_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3935904 INFO (zkCallback-32976-thread-1-processing-n:127.0.0.1:38150_solr) [n:127.0.0.1:38150_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3935912 INFO (jetty-launcher-32956-thread-3) [n:127.0.0.1:45096_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45096_solr
[junit4] 2> 3935914 INFO (zkCallback-32977-thread-1-processing-n:127.0.0.1:42928_solr) [n:127.0.0.1:42928_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3935917 INFO (zkCallback-32974-thread-1-processing-n:127.0.0.1:45096_solr) [n:127.0.0.1:45096_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3935917 INFO (zkCallback-32976-thread-1-processing-n:127.0.0.1:38150_solr) [n:127.0.0.1:38150_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3935918 INFO (zkCallback-32970-thread-1-processing-n:127.0.0.1:47746_solr) [n:127.0.0.1:47746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3935919 INFO (jetty-launcher-32956-thread-1) [n:127.0.0.1:42928_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/tempDir-001/node1/.
[junit4] 2> 3935924 INFO (jetty-launcher-32956-thread-4) [n:127.0.0.1:38150_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/tempDir-001/node4/.
[junit4] 2> 3935931 INFO (jetty-launcher-32956-thread-2) [n:127.0.0.1:47746_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47746_solr
[junit4] 2> 3935933 INFO (zkCallback-32974-thread-1-processing-n:127.0.0.1:45096_solr) [n:127.0.0.1:45096_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3935933 INFO (zkCallback-32977-thread-1-processing-n:127.0.0.1:42928_solr) [n:127.0.0.1:42928_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3935933 INFO (zkCallback-32976-thread-1-processing-n:127.0.0.1:38150_solr) [n:127.0.0.1:38150_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3935933 INFO (zkCallback-32970-thread-1-processing-n:127.0.0.1:47746_solr) [n:127.0.0.1:47746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3935934 INFO (jetty-launcher-32956-thread-3) [n:127.0.0.1:45096_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/tempDir-001/node3/.
[junit4] 2> 3935943 INFO (jetty-launcher-32956-thread-2) [n:127.0.0.1:47746_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_F1F1B13441B49802-001/tempDir-001/node2/.
[junit4] 2> 3935949 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 3936005 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 3936011 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3936012 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3936027 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_41185_hdfs____.gfdi4l/webapp
[junit4] 2> 3936465 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41185
[junit4] 2> 3936549 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3936550 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3936565 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_50691_datanode____xzbwe2/webapp
[junit4] 2> 3936998 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50691
[junit4] 2> 3937057 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3937058 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3937074 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[F1F1B13441B49802]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46434_datanode____hprwhm/webapp
[junit4] 2> 3937118 INFO (IPC Server handler 2 on 47817) [ ] BlockStateChange BLOCK* processReport: from storage DS-bdba0659-6e89-45ec-afb4-8a35fea83d07 node DatanodeRegistration(127.0.0.1:46629, datanodeUuid=54ed237c-cd41-458d-853d-ac2422c1c327, infoPort=49925, infoSecurePort=0, ipcPort=48982, storageInfo=lv=-56;cid=testClusterID;nsid=1240184081;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 3937118 INFO (IPC Server handler 2 on 47817) [ ] BlockStateChange BLOCK* processReport: from storage DS-63fa5
[...truncated too long message...]
e(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 4433886 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[F1F1B13441B49802]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.core.HdfsDirectoryFactoryTest_F1F1B13441B49802-001
[junit4] 2> Nov 02, 2016 12:22:50 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1270, maxMBSortInHeap=5.94377740514397, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=es-SV, timezone=Africa/Djibouti
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (32-bit)/cpus=4,threads=2,free=315935872,total=513277952
[junit4] 2> NOTE: All tests run in this JVM: [UnloadDistributedZkTest, HdfsChaosMonkeySafeLeaderTest, MinimalSchemaTest, TestNamedUpdateProcessors, TestReplicaProperties, DOMUtilTest, TestSort, OverriddenZkACLAndCredentialsProvidersTest, SimpleCollectionCreateDeleteTest, CdcrBootstrapTest, RequestLoggingTest, PingRequestHandlerTest, TestCustomDocTransformer, TestFieldSortValues, DeleteInactiveReplicaTest, DistributedDebugComponentTest, TestStressReorder, TestSlowCompositeReaderWrapper, TestSolrCloudWithDelegationTokens, OpenExchangeRatesOrgProviderTest, CoreAdminHandlerTest, RollingRestartTest, DataDrivenBlockJoinTest, TestInitParams, TestLFUCache, TestCollapseQParserPlugin, TestSolrCLIRunExample, UpdateParamsTest, HdfsRecoverLeaseTest, TestShardHandlerFactory, TestHighFrequencyDictionaryFactory, TestDocTermOrds, TestConfigSets, CoreSorterTest, TestAnalyzedSuggestions, DirectUpdateHandlerTest, TestCloudInspectUtil, DocValuesMultiTest, SuggesterWFSTTest, TestWriterPerf, CloneFieldUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestManagedSynonymFilterFactory, MigrateRouteKeyTest, TestDistributedMissingSort, StatelessScriptUpdateProcessorFactoryTest, TestQueryWrapperFilter, TestPKIAuthenticationPlugin, TestUtils, TestSolrQueryParser, ParsingFieldUpdateProcessorsTest, ExplicitHLLTest, TestZkChroot, OverseerTest, TestTrackingShardHandlerFactory, DistributedQueryComponentOptimizationTest, TestSolrConfigHandlerCloud, CdcrRequestHandlerTest, HdfsUnloadDistributedZkTest, DistribCursorPagingTest, BlockJoinFacetSimpleTest, OverseerRolesTest, BasicDistributedZk2Test, TestDynamicLoading, DirectUpdateHandlerOptimizeTest, TestXIncludeConfig, TestCollationField, TestCloudManagedSchema, NoCacheHeaderTest, JSONWriterTest, PeerSyncReplicationTest, LukeRequestHandlerTest, HttpPartitionTest, TestCryptoKeys, TestSolrJ, TestFieldCacheSanityChecker, ReplaceNodeTest, TestCoreDiscovery, PluginInfoTest, DistributedFacetPivotLargeTest, CollectionsAPISolrJTest, TestExactSharedStatsCache, TestSearcherReuse, TestConfigSetsAPI, TestLMJelinekMercerSimilarityFactory, XmlUpdateRequestHandlerTest, PrimitiveFieldTypeTest, BadIndexSchemaTest, SimpleMLTQParserTest, BasicAuthIntegrationTest, ConnectionManagerTest, AssignTest, SortSpecParsingTest, PrimUtilsTest, TestExtendedDismaxParser, TestReversedWildcardFilterFactory, TestSchemaVersionResource, TestScoreJoinQPScore, TestConfig, SuggesterTest, FieldMutatingUpdateProcessorTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestRandomDVFaceting, ZkCLITest, AutoCommitTest, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestJoin, TestReload, TestStressVersions, HardAutoCommitTest, DistributedTermsComponentTest, SimpleFacetsTest, StatsComponentTest, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, ConvertedLegacyTest, TestFunctionQuery, TestBadConfig, TestIndexSearcher, SoftAutoCommitTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTSTTest, TestStressLucene, TestCSVLoader, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestAtomicUpdateErrorCases, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, DebugComponentTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, FileBasedSpellCheckerTest, RequestHandlersTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, IndexSchemaTest, TestJmxIntegration, JSONWriterTest, TestCSVResponseWriter, MBeansHandlerTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, TestSolrIndexConfig, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestBinaryField, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, TimeZoneUtilsTest, ResourceLoaderTest, TestFastLRUCache, PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, DateFieldTest, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, DistributedIntervalFacetingTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest, ConcurrentDeleteAndCreateCollectionTest, CreateCollectionCleanupTest, CustomCollectionTest, DeleteNodeTest, DeleteReplicaTest, DeleteStatusTest, DistribJoinFromCollectionTest, DistributedQueueTest, LeaderFailureAfterFreshStartTest, OverseerTaskQueueTest, RecoveryAfterSoftCommitTest, ReplicationFactorTest, RestartWhileUpdatingTest, SharedFSAutoReplicaFailoverTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudRecovery, TestClusterProperties, TestCollectionAPI, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLocalFSCloudBackupRestore, TestRandomFlRTGCloud, TlogReplayBufferedWhileIndexingTest, HdfsSyncSliceTest, HdfsWriteToMultipleCollectionsTest, TestClusterStateMutator, ZkStateReaderTest, ZkStateWriterTest, RuleEngineTest, RulesTest, BlobRepositoryCloudTest, CachingDirectoryFactoryTest, HdfsDirectoryFactoryTest]
[junit4] Completed [519/648 (3!)] on J2 in 13.00s, 3 tests, 1 error <<< FAILURES!
[...truncated 60779 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 190 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/190/
4 tests failed.
FAILED: org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([A2675433BE91738A]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterWithThreads
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([A2675433BE91738A]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={ "replicationFactor":"3", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/2)={
"replicationFactor":"3",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([8DBDC023EAAA0C40:C5C8B497EC9923D5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([8DBDC023EAAA0C40:A0C38F791D8EB64C]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 1631 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads
[junit4] 2> أكت 31, 2016 2:14:21 م com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexWriterWithThreads
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=104, name=Thread-54, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 4) Thread[id=103, name=Thread-53, state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.util.ComparableTimSort.sort(ComparableTimSort.java:188)
[junit4] 2> at java.util.Arrays.sort(Arrays.java:1246)
[junit4] 2> at org.apache.lucene.store.RAMDirectory.listAll(RAMDirectory.java:128)
[junit4] 2> at org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:100)
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.sizeInBytes(MockDirectoryWrapper.java:261)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.checkDiskFull(MockIndexOutputWrapper.java:55)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:133)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:72)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeLong(DataOutput.java:213)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeCRC(CodecUtil.java:546)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:390)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsIndexWriter.finish(CompressingStoredFieldsIndexWriter.java:205)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finish(CompressingStoredFieldsWriter.java:467)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:116)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:443)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:396)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:499)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 5) Thread[id=73, name=TEST-TestIndexWriterWithThreads.testImmediateDiskFullWithThreads-seed#[A2675433BE91738A], state=WAITING, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:163)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 6) Thread[id=105, name=Thread-55, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 7) Thread[id=72, name=SUITE-TestIndexWriterWithThreads-seed#[A2675433BE91738A], state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterWithThreads -Dtests.method=testImmediateDiskFullWithThreads -Dtests.seed=A2675433BE91738A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-MA -Dtests.timezone=America/Manaus -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7192s J0 | TestIndexWriterWithThreads.testImmediateDiskFullWithThreads <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([A2675433BE91738A]:0)
[junit4] 2> أكت 31, 2016 2:14:21 م com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 4 leaked thread(s).
[junit4] 2> أكت 31, 2016 2:14:42 م com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 4 threads leaked from SUITE scope at org.apache.lucene.index.TestIndexWriterWithThreads:
[junit4] 2> 1) Thread[id=104, name=Thread-54, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 2) Thread[id=103, name=Thread-53, state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.util.ComparableTimSort.sort(ComparableTimSort.java:188)
[junit4] 2> at java.util.Arrays.sort(Arrays.java:1246)
[junit4] 2> at org.apache.lucene.store.RAMDirectory.listAll(RAMDirectory.java:128)
[junit4] 2> at org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:100)
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.sizeInBytes(MockDirectoryWrapper.java:261)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.checkDiskFull(MockIndexOutputWrapper.java:55)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:133)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:72)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeLong(DataOutput.java:213)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeCRC(CodecUtil.java:546)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:390)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsIndexWriter.finish(CompressingStoredFieldsIndexWriter.java:205)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finish(CompressingStoredFieldsWriter.java:467)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:116)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:443)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:396)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:499)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 3) Thread[id=73, name=TEST-TestIndexWriterWithThreads.testImmediateDiskFullWithThreads-seed#[A2675433BE91738A], state=WAITING, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads.testImmediateDiskFullWithThreads(TestIndexWriterWithThreads.java:163)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4) Thread[id=105, name=Thread-55, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> أكت 31, 2016 2:14:42 م com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=104, name=Thread-54, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> 2) Thread[id=103, name=Thread-53, state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> 3) Thread[id=73, name=TEST-TestIndexWriterWithThreads.testImmediateDiskFullWithThreads-seed#[A2675433BE91738A], state=WAITING, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> 4) Thread[id=105, name=Thread-55, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/core/test/J0/temp/lucene.index.TestIndexWriterWithThreads_A2675433BE91738A-001
[junit4] 2> Oct 31, 2016 6:14:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=104, name=Thread-54, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 2) Thread[id=103, name=Thread-53, state=RUNNABLE, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at java.util.ComparableTimSort.sort(ComparableTimSort.java:188)
[junit4] 2> at java.util.Arrays.sort(Arrays.java:1246)
[junit4] 2> at org.apache.lucene.store.RAMDirectory.listAll(RAMDirectory.java:128)
[junit4] 2> at org.apache.lucene.store.NRTCachingDirectory.listAll(NRTCachingDirectory.java:100)
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.sizeInBytes(MockDirectoryWrapper.java:261)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.checkDiskFull(MockIndexOutputWrapper.java:55)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:133)
[junit4] 2> at org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:72)
[junit4] 2> at org.apache.lucene.store.DataOutput.writeLong(DataOutput.java:213)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeCRC(CodecUtil.java:546)
[junit4] 2> at org.apache.lucene.codecs.CodecUtil.writeFooter(CodecUtil.java:390)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsIndexWriter.finish(CompressingStoredFieldsIndexWriter.java:205)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.finish(CompressingStoredFieldsWriter.java:467)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:116)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:443)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:396)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:499)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> 3) Thread[id=105, name=Thread-55, state=BLOCKED, group=TGRP-TestIndexWriterWithThreads]
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:642)
[junit4] 2> at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
[junit4] 2> at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.<init>(CompressingTermVectorsWriter.java:221)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsFormat.vectorsWriter(CompressingTermVectorsFormat.java:98)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.initTermVectorsWriter(TermVectorsConsumer.java:88)
[junit4] 2> at org.apache.lucene.index.TermVectorsConsumer.finishDocument(TermVectorsConsumer.java:103)
[junit4] 2> at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:93)
[junit4] 2> at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:389)
[junit4] 2> at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:231)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:478)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1562)
[junit4] 2> at org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:86)
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=18150, maxDocsPerChunk=169, blockSize=9), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=18150, blockSize=9)), sim=ClassicSimilarity, locale=ar-MA, timezone=America/Manaus
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (32-bit)/cpus=4,threads=4,free=185966848,total=375390208
[junit4] 2> NOTE: All tests run in this JVM: [TestParallelCompositeReader, TestCrash, TestUpgradeIndexMergePolicy, TestDocInverterPerFieldErrorInfo, TestSpanTermQuery, TestDeterminism, TestTermVectorsReader, TestFilterLeafReader, TestExceedMaxTermLength, TestForTooMuchCloning, TestDocValuesIndexing, TestRegexpQuery, TestOfflineSorter, TestThreadedForceMerge, FiniteStringsIteratorTest, TestConstantScoreQuery, TestOperations, TestFastCompressionMode, TestIndexWriterWithThreads]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterWithThreads -Dtests.seed=A2675433BE91738A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-MA -Dtests.timezone=America/Manaus -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestIndexWriterWithThreads (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([A2675433BE91738A]:0)
[junit4] Completed [386/440 (1!)] on J0 in 7223.23s, 9 tests, 2 errors <<< FAILURES!
[...truncated 9713 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/init-core-data-001
[junit4] 2> 247133 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 247134 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001
[junit4] 2> 247134 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 247134 INFO (Thread-1079) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 247134 INFO (Thread-1079) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 247234 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:60582
[junit4] 2> 247248 INFO (jetty-launcher-272-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 247250 INFO (jetty-launcher-272-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 247250 INFO (jetty-launcher-272-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 247261 INFO (jetty-launcher-272-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 247263 INFO (jetty-launcher-272-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f88c2fe{/solr,null,AVAILABLE}
[junit4] 2> 247265 INFO (jetty-launcher-272-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@67b51609{HTTP/1.1,[http/1.1]}{127.0.0.1:39314}
[junit4] 2> 247265 INFO (jetty-launcher-272-thread-1) [ ] o.e.j.s.Server Started @250647ms
[junit4] 2> 247265 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39314}
[junit4] 2> 247268 ERROR (jetty-launcher-272-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 247268 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 247268 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 247268 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 247268 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-31T20:13:30.072Z
[junit4] 2> 247278 INFO (jetty-launcher-272-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a141758{/solr,null,AVAILABLE}
[junit4] 2> 247278 INFO (jetty-launcher-272-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@96a7e90{HTTP/1.1,[http/1.1]}{127.0.0.1:35949}
[junit4] 2> 247278 INFO (jetty-launcher-272-thread-2) [ ] o.e.j.s.Server Started @250660ms
[junit4] 2> 247278 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35949}
[junit4] 2> 247278 ERROR (jetty-launcher-272-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 247279 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 247279 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 247279 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 247279 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-31T20:13:30.083Z
[junit4] 2> 247294 INFO (jetty-launcher-272-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b8e7dca{/solr,null,AVAILABLE}
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@28fd5475{HTTP/1.1,[http/1.1]}{127.0.0.1:38724}
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.e.j.s.Server Started @250676ms
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38724}
[junit4] 2> 247295 ERROR (jetty-launcher-272-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 247295 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 247296 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-31T20:13:30.099Z
[junit4] 2> 247306 INFO (jetty-launcher-272-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26e75a83{/solr,null,AVAILABLE}
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@57a68c40{HTTP/1.1,[http/1.1]}{127.0.0.1:35667}
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.e.j.s.Server Started @250688ms
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35667}
[junit4] 2> 247307 ERROR (jetty-launcher-272-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.4.0
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 247307 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-31T20:13:30.111Z
[junit4] 2> 247321 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 247322 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 247322 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 247330 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1581c6019d40002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 247333 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 247337 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 247340 INFO (jetty-launcher-272-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60582/solr
[junit4] 2> 247343 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 247352 INFO (jetty-launcher-272-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60582/solr
[junit4] 2> 247357 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 247358 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 247364 INFO (jetty-launcher-272-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60582/solr
[junit4] 2> 247365 INFO (jetty-launcher-272-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60582/solr
[junit4] 2> 247378 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1581c6019d40006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 247380 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1581c6019d40007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 247444 INFO (jetty-launcher-272-thread-1) [n:127.0.0.1:39314_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39314_solr
[junit4] 2> 247446 INFO (jetty-launcher-272-thread-4) [n:127.0.0.1:38724_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38724_solr
[junit4] 2> 247446 INFO (jetty-launcher-272-thread-1) [n:127.0.0.1:39314_solr ] o.a.s.c.Overseer Overseer (id=96858591064227849-127.0.0.1:39314_solr-n_0000000000) starting
[junit4] 2> 247447 INFO (jetty-launcher-272-thread-3) [n:127.0.0.1:35667_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35667_solr
[junit4] 2> 247448 INFO (zkCallback-293-thread-1-processing-n:127.0.0.1:35949_solr) [n:127.0.0.1:35949_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 247449 INFO (zkCallback-290-thread-1-processing-n:127.0.0.1:39314_solr) [n:127.0.0.1:39314_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 247450 INFO (zkCallback-291-thread-1-processing-n:127.0.0.1:38724_solr) [n:127.0.0.1:38724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 247454 INFO (zkCallback-291-thread-1-processing-n:127.0.0.1:38724_solr) [n:127.0.0.1:38724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 247454 INFO (zkCallback-290-thread-1-processing-n:127.0.0.1:39314_solr) [n:127.0.0.1:39314_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 247454 INFO (zkCallback-293-thread-1-processing-n:127.0.0.1:35949_solr) [n:127.0.0.1:35949_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 247469 INFO (zkCallback-292-thread-1-processing-n:127.0.0.1:35667_solr) [n:127.0.0.1:35667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 247473 INFO (zkCallback-292-thread-2-processing-n:127.0.0.1:35667_solr) [n:127.0.0.1:35667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 247475 INFO (jetty-launcher-272-thread-2) [n:127.0.0.1:35949_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35949_solr
[junit4] 2> 247481 INFO (zkCallback-293-thread-1-processing-n:127.0.0.1:35949_solr) [n:127.0.0.1:35949_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 247482 INFO (zkCallback-291-thread-1-processing-n:127.0.0.1:38724_solr) [n:127.0.0.1:38724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 247482 INFO (OverseerStateUpdate-96858591064227849-127.0.0.1:39314_solr-n_0000000000) [n:127.0.0.1:39314_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 247487 INFO (zkCallback-292-thread-2-processing-n:127.0.0.1:35667_solr) [n:127.0.0.1:35667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 247488 INFO (jetty-launcher-272-thread-1) [n:127.0.0.1:39314_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39314_solr
[junit4] 2> 247490 INFO (zkCallback-292-thread-2-processing-n:127.0.0.1:35667_solr) [n:127.0.0.1:35667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 247490 INFO (zkCallback-291-thread-1-processing-n:127.0.0.1:38724_solr) [n:127.0.0.1:38724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 247490 INFO (zkCallback-290-thread-1-processing-n:127.0.0.1:39314_solr) [n:127.0.0.1:39314_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 247492 INFO (zkCallback-293-thread-1-processing-n:127.0.0.1:35949_solr) [n:127.0.0.1:35949_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 247496 INFO (jetty-launcher-272-thread-3) [n:127.0.0.1:35667_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node3/.
[junit4] 2> 247510 INFO (jetty-launcher-272-thread-2) [n:127.0.0.1:35949_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node2/.
[junit4] 2> 247514 INFO (jetty-launcher-272-thread-1) [n:127.0.0.1:39314_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node1/.
[junit4] 2> 247514 INFO (jetty-launcher-272-thread-4) [n:127.0.0.1:38724_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node4/.
[junit4] 2> 247543 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 247599 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 247609 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 247610 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 247624 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_53183_hdfs____.s98s08/webapp
[junit4] 2> 248062 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53183
[junit4] 2> 248362 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 248363 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 248376 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_50125_datanode____jgw65c/webapp
[junit4] 2> 248769 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50125
[junit4] 2> 248825 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 248826 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 248845 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41504_datanode____.w43wzh/webapp
[junit4] 2> 248880 INFO (IPC Server handler 4 on 46170) [ ] BlockStateChange BLOCK* processReport: from storage DS-2a1c0edf-59e1-4df8-b477-ab51afe10333 node DatanodeRegistration(127.0.0.1:54284, datanodeUuid=f145fa02-6563-4423-a079-a0098091ca3f, infoPort=55549, infoSecurePort=0, ipcPort=51066, storageInfo=lv=-56;cid=testClusterID;nsid=386242792;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 248880 INFO (IPC Server handler 4 on 46170) [ ] BlockStateChange BLOCK* processReport: from storage DS-e415f201-b606-458a-9c54-0b2cb8e9932c node DatanodeRegistration(127.0.0.1:54284, datanodeUuid=f145fa02-6563-4423-a079-a0098091ca3f, infoPort=55549, infoSecurePort=0, ipcPort=51066, storageInfo=lv=-56;cid=testClusterID;nsid=386242792;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 249254 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41504
[junit4] 2> 249359 INFO (IPC Server handler 3 on 46170) [ ] BlockStateChange BLOCK* processReport: from storage DS-621733c7-f8e3-40fa-8bf0-754d43e01335 node DatanodeRegistration(127.0.0.1:38521, datanodeUuid=9237200b-106f-4008-b994-a8899f5bf161, infoPort=57998, infoSecurePort=0, ipcPort=44175, storageInfo=lv=-56;cid=testClusterID;nsid=386242792;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 249359 INFO (IPC Server handler 3 on 46170) [ ] BlockStateChange BLOCK* processReport: from storage DS-91554b72-8e70-47a3-a6f6-32e2bf3dfd87 node DatanodeRegistration(127.0.0.1:38521, datanodeUuid=9237200b-106f-4008-b994-a8899f5bf161, infoPort=57998, infoSecurePort=0, ipcPort=44175, storageInfo=lv=-56;cid=testClusterID;nsid=386242792;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 249465 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[8DBDC023EAAA0C40]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCoresAreDistributedAcrossNodes
[junit4] 2> 249467 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.testCoresAreDistributedAcrossNodes-seed#[8DBDC023EAAA0C40]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 249477 INFO (qtp1124549653-2322) [n:127.0.0.1:39314_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=nodes_used_collection&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 249483 INFO (OverseerThreadFactory-576-thread-1-processing-n:127.0.0.1:39314_solr) [n:127.0.0.1:39314_solr ] o.a.s.c.CreateCollectionCmd Create collection nodes_used_collection
[junit4] 2> 249600 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 249606 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard2_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 249607 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica2&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 249609 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=nodes_used_collection_shard1_replica1&action=CREATE&numShards=2&collection=nodes_used_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 249718 INFO (zkCallback-293-thread-1-processing-n:127.0.0.1:35949_solr) [n:127.0.0.1:35949_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 249718 INFO (zkCallback-292-thread-2-processing-n:127.0.0.1:35667_solr) [n:127.0.0.1:35667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 249718 INFO (zkCallback-291-thread-1-processing-n:127.0.0.1:38724_solr) [n:127.0.0.1:38724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 249718 INFO (zkCallback-290-thread-4-processing-n:127.0.0.1:39314_solr) [n:127.0.0.1:39314_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/nodes_used_collection/state.json] for collection [nodes_used_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 250623 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 250633 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 250635 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 250636 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica1] Schema name=minimal
[junit4] 2> 250640 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 250642 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 250643 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.4.0
[junit4] 2> 250650 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica1] Schema name=minimal
[junit4] 2> 250653 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 250654 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46170/data
[junit4] 2> 250654 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 250654 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard2_replica2] Schema name=minimal
[junit4] 2> 250654 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node1/nodes_used_collection_shard2_replica1], dataDir=[hdfs://localhost:46170/data/nodes_used_collection/core_node1/data/]
[junit4] 2> 250654 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node1/data/snapshot_metadata
[junit4] 2> 250658 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica1' using configuration from collection nodes_used_collection
[junit4] 2> 250659 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46170/data
[junit4] 2> 250659 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 250659 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node2/nodes_used_collection_shard1_replica1], dataDir=[hdfs://localhost:46170/data/nodes_used_collection/core_node4/data/]
[junit4] 2> 250660 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node4/data/snapshot_metadata
[junit4] 2> 250661 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema [nodes_used_collection_shard1_replica2] Schema name=minimal
[junit4] 2> 250665 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 250688 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 250688 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 250688 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 250691 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 250691 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 x:nodes_used_collection_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard2_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 250692 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46170/data
[junit4] 2> 250692 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 250692 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node3/nodes_used_collection_shard2_replica2], dataDir=[hdfs://localhost:46170/data/nodes_used_collection/core_node2/data/]
[junit4] 2> 250692 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node2/data/snapshot_metadata
[junit4] 2> 250699 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 250699 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 250699 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 250700 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 x:nodes_used_collection_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'nodes_used_collection_shard1_replica2' using configuration from collection nodes_used_collection
[junit4] 2> 250701 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:46170/data
[junit4] 2> 250701 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 250701 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.SolrCore [[nodes_used_collection_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-001/node4/nodes_used_collection_shard1_replica2], dataDir=[hdfs://localhost:46170/data/nodes_used_collection/core_node3/data/]
[junit4] 2> 250701 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node3/data/snapshot_metadata
[junit4] 2> 250711 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 250711 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 250711 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 250725 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 250725 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 250725 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 250735 WARN (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 250737 WARN (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 250740 WARN (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 250748 WARN (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 255740 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255743 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255743 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node1/data
[junit4] 2> 255744 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node4/data
[junit4] 2> 255747 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255747 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node3/data
[junit4] 2> 255756 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255759 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node2/data
[junit4] 2> 255775 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node1/data/index
[junit4] 2> 255782 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node3/data/index
[junit4] 2> 255782 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node4/data/index
[junit4] 2> 255783 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 255783 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 255783 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 255789 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 255789 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 255789 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 255796 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 255796 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 255796 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 255809 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:46170/data/nodes_used_collection/core_node2/data/index
[junit4] 2> 255825 INFO (qtp206216247-2331) [n:127.0.0.1:35949_solr c:nodes_used_collection s:shard1 r:core_node4 x:nodes_used_collection_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255828 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 255828 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 255828 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 255830 INFO (qtp1124549653-2319) [n:127.0.0.1:39314_solr c:nodes_used_collection s:shard2 r:core_node1 x:nodes_used_collection_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255832 INFO (qtp1165959295-2328) [n:127.0.0.1:38724_solr c:nodes_used_collection s:shard1 r:core_node3 x:nodes_used_collection_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255849 INFO (qtp518115045-2350) [n:127.0.0.1:35667_solr c:nodes_used_collection s:shard2 r:core_node2 x:nodes_used_collection_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 255892 INFO (IPC Server handler 5 on 46170) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38521 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-2a1c0edf-59e1-4df8
[...truncated too long message...]
empDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:46170) [ ] o.a.h.h.s.d.DataNode BPOfferService for Block pool BP-1908895843-10.41.0.5-1477944810377 (Datanode Uuid f145fa02-6563-4423-a079-a0098091ca3f) service to localhost/127.0.0.1:46170 interrupted
[junit4] 2> 407695 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:46170) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1908895843-10.41.0.5-1477944810377 (Datanode Uuid f145fa02-6563-4423-a079-a0098091ca3f) service to localhost/127.0.0.1:46170
[junit4] 2> 407705 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 407809 ERROR (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
[junit4] 2> java.lang.reflect.InvocationTargetException
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 407813 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60582 60582
[junit4] 2> 407814 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DBDC023EAAA0C40]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DBDC023EAAA0C40-001
[junit4] 2> Oct 31, 2016 8:16:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=230, maxMBSortInHeap=7.623337626798657, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=ca-ES, timezone=Europe/Vienna
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (32-bit)/cpus=4,threads=2,free=317202352,total=484966400
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCloudSnapshots, TestIBSimilarityFactory, TestRawTransformer, ScriptEngineTest, TestCorePropertiesReload, TestFileDictionaryLookup, TestFieldTypeCollectionResource, DistributedQueryElevationComponentTest, HdfsDirectoryTest, HdfsWriteToMultipleCollectionsTest, TestPerFieldSimilarity, TestSolrFieldCacheMBean, TestSQLHandler, TestLockTree, DeleteLastCustomShardedReplicaTest, TestOmitPositions, TestReplicationHandlerBackup, TestFastWriter, ReturnFieldsTest, TestPseudoReturnFields, TestClassicSimilarityFactory, FullHLLTest, TestInitQParser, TestRealTimeGet, ResourceLoaderTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [40/648 (1!)] on J0 in 168.81s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 62303 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 189 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/189/
6 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest.test
Error Message:
document count mismatch. control=2045 sum(shards)=2008 cloudClient=2008
Stack Trace:
java.lang.AssertionError: document count mismatch. control=2045 sum(shards)=2008 cloudClient=2008
at __randomizedtesting.SeedInfo.seed([BDF06599CB58DC72:35A45A4365A4B18A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1323)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:228)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler
Error Message:
ObjectTracker found 9 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 9 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([BDF06599CB58DC72]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:260)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.lucene.search.TestFuzzyQuery.testRandom
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([77127CBC73D9BF60]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([77127CBC73D9BF60]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([BDF06599CB58DC72:908E2AC33C7C667E]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={ "replicationFactor":"4", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"3", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/4)={
"replicationFactor":"4",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"3",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([BDF06599CB58DC72:F585112DCD6BF3E7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 1828 lines...]
[junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> okt. 31, 2016 7:46:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> 1) Thread[id=3550, name=SUITE-TestFuzzyQuery-seed#[77127CBC73D9BF60], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 2) Thread[id=3551, name=TEST-TestFuzzyQuery.testRandom-seed#[77127CBC73D9BF60], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Object.hashCode(Native Method)
[junit4] 2> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
[junit4] 2> at java.util.WeakHashMap.get(WeakHashMap.java:396)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.getPerThread(RandomizedContext.java:319)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.getRandomness(RandomizedContext.java:92)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.getRandom(RandomizedContext.java:114)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.method=testRandom -Dtests.seed=77127CBC73D9BF60 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=sl-SI -Dtests.timezone=Antarctica/Macquarie -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7200s J2 | TestFuzzyQuery.testRandom <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([77127CBC73D9BF60]:0)
[junit4] 2> okt. 31, 2016 7:46:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> okt. 31, 2016 7:46:21 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestFuzzyQuery:
[junit4] 2> 1) Thread[id=3551, name=TEST-TestFuzzyQuery.testRandom-seed#[77127CBC73D9BF60], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> okt. 31, 2016 7:46:21 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=3551, name=TEST-TestFuzzyQuery.testRandom-seed#[77127CBC73D9BF60], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> okt. 31, 2016 7:46:24 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=3551, name=TEST-TestFuzzyQuery.testRandom-seed#[77127CBC73D9BF60], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Object.hashCode(Native Method)
[junit4] 2> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:426)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {field=PostingsFormat(name=Asserting)}, docValues:{}, maxPointsInLeafNode=1509, maxMBSortInHeap=5.466130839774704, sim=ClassicSimilarity, locale=sl-SI, timezone=Antarctica/Macquarie
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=213838800,total=503316480
[junit4] 2> NOTE: All tests run in this JVM: [TestBlockPostingsFormat3, TestSloppyMath, TestNIOFSDirectory, TestSameScoresWithThreads, TestDocBoost, TestBufferedIndexInput, TestUsageTrackingFilterCachingPolicy, TestMultiLevelSkipList, TestFixedLengthBytesRefArray, Test2BPagedBytes, TestIndexFileDeleter, TestMinShouldMatch2, TestSortedSetDocValues, TestSortRescorer, TestSpanOrQuery, TestParallelTermEnum, TestTermVectors, Test2BPostings, TestCharTermAttributeImpl, TestNoDeletionPolicy, TestTermsEnum2, TestVersion, TestRollingUpdates, Test2BBKDPoints, TestNorms, TestIndexWriterMerging, TestUTF32ToUTF8, TestTimSorterWorstCase, TestExternalCodecs, TestMultiMMap, TestForceMergeForever, TestSortedSetSortField, TestSentinelIntSet, TestMaxPosition, Test2BSortedDocValuesOrds, TestCollectionUtil, TestCharacterUtils, TestMergeSchedulerExternal, TestLucene60FieldInfoFormat, TestWildcard, TestNRTCachingDirectory, TestIntsRef, TestCharsRefBuilder, TestQueryRescorer, TestComplexExplanationsOfNonMatches, TestNumericDocValuesUpdates, TestIndexSorting, TestMultiTermConstantScore, TestTieredMergePolicy, TestFastDecompressionMode, TestInfoStream, TestControlledRealTimeReopenThread, TestTopDocsCollector, TestGrowableByteArrayDataOutput, TestDirectPacked, TestTryDelete, TestToken, TestIntArrayDocIdSet, TestSloppyPhraseQuery, TestDisjunctionMaxQuery, TestIndexCommit, TestDirectoryReader, TestIndexWriterReader, TestBagOfPositions, TestVirtualMethod, TestFileSwitchDirectory, TestPolygon2D, TestBytesStore, TestCodecHoldsOpenFiles, TestMathUtil, TestLazyProxSkipping, TestIndexWriterFromReader, TestTermVectorsWriter, TestNeverDelete, TestTopDocsMerge, TestLongPostings, TestMinimize, TestIndexWriterForceMerge, TestArrayUtil, TestByteSlices, TestConsistentFieldNumbers, TestPhraseQuery, TestOmitNorms, TestIndexWriterNRTIsCurrent, TestFieldCacheRewriteMethod, TestTransactions, TestCrash, TestMultiThreadTermVectors, TestSort, TestFieldsReader, TestCustomSearcherSort, TestDocValuesIndexing, TestSimilarityBase, TestBooleanMinShouldMatch, TestLegacyNumericUtils, Test2BDocs, TestIndexWriterOnDiskFull, TestSpans, TestMixedCodecs, TestTransactionRollback, TestFlex, TestSpanExplanations, TestStressIndexing2, TestMultiPhraseQuery, TestCodecs, TestParallelLeafReader, TestBytesRefHash, TestStressNRT, TestIndexInput, TestSloppyPhraseQuery2, TestNearSpansOrdered, TestMultiFields, TestTermRangeQuery, TestDocumentsWriterDeleteQueue, TestDocumentWriter, TestLiveFieldValues, TestForTooMuchCloning, TestSpanSearchEquivalence, TestFuzzyQuery]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.seed=77127CBC73D9BF60 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=sl-SI -Dtests.timezone=Antarctica/Macquarie -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | TestFuzzyQuery (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([77127CBC73D9BF60]:0)
[junit4] Completed [440/440 (1!)] on J2 in 7223.13s, 5 tests, 2 errors <<< FAILURES!
[...truncated 9770 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/init-core-data-001
[junit4] 2> 659546 INFO (SUITE-TestReplicationHandler-seed#[BDF06599CB58DC72]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 659552 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestRepeater
[junit4] 2> 659557 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/collection1
[junit4] 2> 659569 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 659582 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ea216c3{/solr,null,AVAILABLE}
[junit4] 2> 659585 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.ServerConnector Started ServerConnector@16fca170{HTTP/1.1,[http/1.1]}{127.0.0.1:60207}
[junit4] 2> 659585 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server Started @663404ms
[junit4] 2> 659585 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=60207}
[junit4] 2> 659586 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 659586 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 659586 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 659587 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 659587 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-30T22:55:09.290Z
[junit4] 2> 659587 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/solr.xml
[junit4] 2> 659595 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 659598 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/.
[junit4] 2> 659598 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 659623 WARN (coreLoadExecutor-767-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 659626 INFO (coreLoadExecutor-767-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 659650 INFO (coreLoadExecutor-767-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 659656 INFO (coreLoadExecutor-767-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 659663 INFO (coreLoadExecutor-767-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/./collection1
[junit4] 2> 659663 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/./collection1/data/]
[junit4] 2> 659670 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=20.6201171875, floorSegmentMB=0.3173828125, forceMergeDeletesPctAllowed=28.610260248179923, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12318225173149289
[junit4] 2> 659685 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 659685 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 659689 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=16.8681640625, floorSegmentMB=1.021484375, forceMergeDeletesPctAllowed=25.82892615583165, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 659690 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@8b1dee2[collection1] main]
[junit4] 2> 659690 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-001/collection1/conf
[junit4] 2> 659691 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 659691 INFO (coreLoadExecutor-767-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 659691 INFO (searcherExecutor-768-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@8b1dee2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 660107 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1
[junit4] 2> 660111 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 660121 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3d51ad6f{/solr,null,AVAILABLE}
[junit4] 2> 660122 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.ServerConnector Started ServerConnector@78d023c4{HTTP/1.1,[http/1.1]}{127.0.0.1:45073}
[junit4] 2> 660122 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server Started @663941ms
[junit4] 2> 660122 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=45073}
[junit4] 2> 660122 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 660123 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 660123 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 660123 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 660123 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-30T22:55:09.826Z
[junit4] 2> 660123 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/solr.xml
[junit4] 2> 660129 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 660132 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/.
[junit4] 2> 660132 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 660141 WARN (coreLoadExecutor-777-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 660142 INFO (coreLoadExecutor-777-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 660151 INFO (coreLoadExecutor-777-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 660153 INFO (coreLoadExecutor-777-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 660154 INFO (coreLoadExecutor-777-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/./collection1
[junit4] 2> 660155 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/./collection1/data/]
[junit4] 2> 660156 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=20.6201171875, floorSegmentMB=0.3173828125, forceMergeDeletesPctAllowed=28.610260248179923, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12318225173149289
[junit4] 2> 660163 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 660163 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 660163 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=16.8681640625, floorSegmentMB=1.021484375, forceMergeDeletesPctAllowed=25.82892615583165, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 660163 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3675f9ec[collection1] main]
[junit4] 2> 660163 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1/conf
[junit4] 2> 660179 INFO (coreLoadExecutor-777-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 660180 INFO (searcherExecutor-778-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3675f9ec[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 660658 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@78d023c4{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 660658 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1986890419
[junit4] 2> 660659 INFO (coreCloseExecutor-783-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4547e16b
[junit4] 2> 660667 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3d51ad6f{/solr,null,UNAVAILABLE}
[junit4] 2> 660672 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 660675 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5da999f{/solr,null,AVAILABLE}
[junit4] 2> 660676 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.ServerConnector Started ServerConnector@5f18d7bc{HTTP/1.1,[http/1.1]}{127.0.0.1:33206}
[junit4] 2> 660676 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server Started @664495ms
[junit4] 2> 660676 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=33206}
[junit4] 2> 660676 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 660686 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 660687 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 660687 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 660687 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-30T22:55:10.390Z
[junit4] 2> 660687 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/solr.xml
[junit4] 2> 660694 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 660700 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/.
[junit4] 2> 660700 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 660710 WARN (coreLoadExecutor-790-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 660711 INFO (coreLoadExecutor-790-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 660721 INFO (coreLoadExecutor-790-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 660724 INFO (coreLoadExecutor-790-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 660725 INFO (coreLoadExecutor-790-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/./collection1
[junit4] 2> 660725 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/./collection1/data/]
[junit4] 2> 660726 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=20.6201171875, floorSegmentMB=0.3173828125, forceMergeDeletesPctAllowed=28.610260248179923, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12318225173149289
[junit4] 2> 660733 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 660733 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 660734 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=16.8681640625, floorSegmentMB=1.021484375, forceMergeDeletesPctAllowed=25.82892615583165, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 660734 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6dfe0a1b[collection1] main]
[junit4] 2> 660735 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-002/collection1/conf
[junit4] 2> 660735 INFO (coreLoadExecutor-790-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 660738 INFO (searcherExecutor-791-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6dfe0a1b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 661204 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/collection1
[junit4] 2> 661208 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 661222 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@13fb01e5{/solr,null,AVAILABLE}
[junit4] 2> 661223 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.ServerConnector Started ServerConnector@7dce6b92{HTTP/1.1,[http/1.1]}{127.0.0.1:56236}
[junit4] 2> 661226 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.e.j.s.Server Started @665046ms
[junit4] 2> 661226 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=56236}
[junit4] 2> 661227 ERROR (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 661228 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 661228 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 661228 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 661228 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-30T22:55:10.931Z
[junit4] 2> 661229 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/solr.xml
[junit4] 2> 661237 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 661247 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/.
[junit4] 2> 661248 INFO (TEST-TestReplicationHandler.doTestRepeater-seed#[BDF06599CB58DC72]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 661275 WARN (coreLoadExecutor-800-thread-1) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 661276 INFO (coreLoadExecutor-800-thread-1) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 661285 INFO (coreLoadExecutor-800-thread-1) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 661288 INFO (coreLoadExecutor-800-thread-1) [ ] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 661290 INFO (coreLoadExecutor-800-thread-1) [ ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/./collection1
[junit4] 2> 661291 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/./collection1/data/]
[junit4] 2> 661291 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=3, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=20.6201171875, floorSegmentMB=0.3173828125, forceMergeDeletesPctAllowed=28.610260248179923, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12318225173149289
[junit4] 2> 661319 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 661319 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 661319 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=16.8681640625, floorSegmentMB=1.021484375, forceMergeDeletesPctAllowed=25.82892615583165, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 661320 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@12ed04fc[collection1] main]
[junit4] 2> 661320 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_BDF06599CB58DC72-001/solr-instance-003/collection1/conf
[junit4] 2> 661320 WARN (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 661321 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler No value set for 'pollInterval'. Timer Task not started.
[junit4] 2> 661321 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 661321 INFO (coreLoadExecutor-800-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 661323 INFO (searcherExecutor-801-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@12ed04fc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 661775 INFO (qtp1861330665-49301) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 1
[junit4] 2> 661777 INFO (qtp1861330665-49302) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 661779 INFO (qtp1861330665-49303) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 661781 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 661781 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@66ad5675
[junit4] 2> 661784 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a09af4[collection1] main]
[junit4] 2> 661784 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 661784 INFO (searcherExecutor-768-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5a09af4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.3.0):c3)))}
[junit4] 2> 661785 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
[junit4] 2> 661803 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 661807 INFO (qtp1861330665-49299) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 661807 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 661807 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1477868111484
[junit4] 2> 661807 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 661807 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 661812 INFO (qtp1861330665-49299) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=3
[junit4] 2> 661812 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 4
[junit4] 2> 661819 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=896079736, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 661821 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 661822 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3540739a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f84c55; maxCacheMB=0.1484375 maxMergeSizeMB=0.3193359375))
[junit4] 2> 661836 INFO (qtp1861330665-49301) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.cfe&checksum=true&wt=filestream&command=filecontent} status=0 QTime=5
[junit4] 2> 661847 INFO (qtp1861330665-49302) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 661850 INFO (qtp1861330665-49303) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.cfs&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 661853 INFO (qtp1861330665-49304) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 661854 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=1947) : 0 secs (null bytes/sec) to MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@3540739a lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11f84c55; maxCacheMB=0.1484375 maxMergeSizeMB=0.3193359375))
[junit4] 2> 661857 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=22, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.20766882188399613]
[junit4] 2> 661858 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 661859 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a1e1ed4[collection1] main]
[junit4] 2> 661859 INFO (searcherExecutor-801-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2a1e1ed4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.3.0):c3)))}
[junit4] 2> 661859 INFO (qtp447053692-49354) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication par
[...truncated too long message...]
001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:53038) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-612198244-10.41.0.5-1477870990279 (Datanode Uuid b012931d-df27-496c-ba3a-9227e24cb3c0) service to localhost/127.0.0.1:53038
[junit4] 2> 3701449 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[BDF06599CB58DC72]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3701556 ERROR (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[BDF06599CB58DC72]-worker) [ ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
[junit4] 2> java.lang.reflect.InvocationTargetException
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 3701562 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[BDF06599CB58DC72]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39688 39688
[junit4] 2> 3701562 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[BDF06599CB58DC72]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_BDF06599CB58DC72-001
[junit4] 2> Oct 30, 2016 11:45:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1237, maxMBSortInHeap=5.787088743996432, sim=ClassicSimilarity, locale=ca-ES, timezone=Etc/GMT+3
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=5,free=370762688,total=521666560
[junit4] 2> NOTE: All tests run in this JVM: [ReplicationFactorTest, QueryEqualityTest, TestRandomDVFaceting, TestDistributedSearch, TestComplexPhraseQParserPlugin, TestTrieFacet, TestSolrConfigHandlerConcurrent, TestMiniSolrCloudClusterSSL, TestDefaultStatsCache, TestOmitPositions, TestJoin, DistribDocExpirationUpdateProcessorTest, TestReplicationHandler, TestAtomicUpdateErrorCases, DistributedFacetPivotWhiteBoxTest, DistributedVersionInfoTest, OverseerStatusTest, DeleteShardTest, TestImplicitCoreProperties, AnalysisErrorHandlingTest, DateRangeFieldTest, CurrencyFieldOpenExchangeTest, CloudExitableDirectoryReaderTest, LeaderInitiatedRecoveryOnCommitTest, SynonymTokenizerTest, UniqFieldsUpdateProcessorFactoryTest, TestDocumentBuilder, TestOrdValues, TestRandomFaceting, TestSubQueryTransformerDistrib, AnalyticsMergeStrategyTest, BlockCacheTest, CollectionTooManyReplicasTest, TestSolrDeletionPolicy2, SolrRequestParserTest, RegexBoostProcessorTest, TestPerFieldSimilarityClassic, TestClassicSimilarityFactory, SyncSliceTest, TestCloudPivotFacet, TestNRTOpen, PolyFieldTest, SegmentsInfoRequestHandlerTest, TestBlobHandler, TestSchemaResource, TestSolrCoreProperties, DistributedSuggestComponentTest, SmileWriterTest, OverseerCollectionConfigSetProcessorTest, BigEndianAscendingWordSerializerTest, AlternateDirectoryTest, DocumentAnalysisRequestHandlerTest, TestStressLucene, TestIBSimilarityFactory, TestRangeQuery, CreateCollectionCleanupTest, TestFieldCacheSort, PKIAuthenticationIntegrationTest, TestSerializedLuceneMatchVersion, CdcrReplicationDistributedZkTest, TestManagedSchema, TestSolrCloudWithKerberosAlt, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [437/648 (3!)] on J0 in 172.03s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 61039 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 188 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/188/
6 tests failed.
FAILED: org.apache.lucene.search.TestFuzzyQuery.testRandom
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([B7170FB6D1C73E2]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([B7170FB6D1C73E2]:0)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([3A2118BED78F59EA:B275276479733412]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test
Error Message:
There are still nodes recoverying - waited for 30 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 30 seconds
at __randomizedtesting.SeedInfo.seed([3A2118BED78F59EA:B275276479733412]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:184)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:862)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1418)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([3A2118BED78F59EA:175F57E420ABE3E6]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/6)={ "replicationFactor":"3", "shards":{ "shard1":{ "range":"80000000-b332ffff", "state":"active", "replicas":{}}, "shard2":{ "range":"b3330000-e665ffff", "state":"active", "replicas":{}}, "shard3":{ "range":"e6660000-1998ffff", "state":"active", "replicas":{}}, "shard4":{ "range":"19990000-4ccbffff", "state":"active", "replicas":{}}, "shard5":{ "range":"4ccc0000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"4", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/6)={
"replicationFactor":"3",
"shards":{
"shard1":{
"range":"80000000-b332ffff",
"state":"active",
"replicas":{}},
"shard2":{
"range":"b3330000-e665ffff",
"state":"active",
"replicas":{}},
"shard3":{
"range":"e6660000-1998ffff",
"state":"active",
"replicas":{}},
"shard4":{
"range":"19990000-4ccbffff",
"state":"active",
"replicas":{}},
"shard5":{
"range":"4ccc0000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"4",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([3A2118BED78F59EA:72546C0AD1BC767F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 1634 lines...]
[junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> Okt 29, 2016 6:52:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=5477, name=SUITE-TestFuzzyQuery-seed#[B7170FB6D1C73E2], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
[junit4] 2> 4) Thread[id=5478, name=TEST-TestFuzzyQuery.testRandom-seed#[B7170FB6D1C73E2], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.method=testRandom -Dtests.seed=B7170FB6D1C73E2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Asia/Beirut -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 7200s J1 | TestFuzzyQuery.testRandom <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([B7170FB6D1C73E2]:0)
[junit4] 2> Okt 29, 2016 6:52:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Okt 29, 2016 6:53:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestFuzzyQuery:
[junit4] 2> 1) Thread[id=5478, name=TEST-TestFuzzyQuery.testRandom-seed#[B7170FB6D1C73E2], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Okt 29, 2016 6:53:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=5478, name=TEST-TestFuzzyQuery.testRandom-seed#[B7170FB6D1C73E2], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> Okt 29, 2016 6:53:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=5478, name=TEST-TestFuzzyQuery.testRandom-seed#[B7170FB6D1C73E2], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Object.hashCode(Native Method)
[junit4] 2> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:426)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:503)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {field=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=686, maxMBSortInHeap=6.5759895502748105, sim=RandomSimilarity(queryNorm=true,coord=yes): {field=DFR I(F)LZ(0.3)}, locale=de-AT, timezone=Asia/Beirut
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=26383424,total=143654912
[junit4] 2> NOTE: All tests run in this JVM: [TestNot, TestBytesRefAttImpl, TestMSBRadixSorter, TestSearchWithThreads, TestEarlyTerminatingSortingCollector, TestCheckIndex, TestSegmentReader, TestSearch, TestLucene54DocValuesFormat, TestPrefixQuery, MultiCollectorTest, TestClassicSimilarity, TestSortRandom, TestIndexableField, TestFieldType, Test2BPositions, TestDemoParallelLeafReader, TestPerFieldDocValuesFormat, TestBasics, TestFuzzyQuery]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.seed=B7170FB6D1C73E2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Asia/Beirut -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | TestFuzzyQuery (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([B7170FB6D1C73E2]:0)
[junit4] Completed [387/440 (1!)] on J1 in 7223.16s, 10 tests, 2 errors <<< FAILURES!
[...truncated 10416 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/init-core-data-001
[junit4] 2> 1229330 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 1229331 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /dr/
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1229425 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1229451 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1229453 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1229473 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_54529_hdfs____.kq1teh/webapp
[junit4] 2> 1229917 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:54529
[junit4] 2> 1230090 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1230092 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1230106 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_55778_datanode____.jpe1t/webapp
[junit4] 2> 1230541 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55778
[junit4] 2> 1230661 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1230663 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1230702 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46540_datanode____.hc1dre/webapp
[junit4] 2> 1230771 INFO (IPC Server handler 0 on 37863) [ ] BlockStateChange BLOCK* processReport: from storage DS-f33145a3-a520-4056-99b8-11ec8d8416c5 node DatanodeRegistration(127.0.0.1:33893, datanodeUuid=79a7a878-9292-4e46-8ee3-6d130a7b9ee3, infoPort=55167, infoSecurePort=0, ipcPort=56591, storageInfo=lv=-56;cid=testClusterID;nsid=446232316;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1230771 INFO (IPC Server handler 0 on 37863) [ ] BlockStateChange BLOCK* processReport: from storage DS-ef2addb3-abd7-4c8f-952b-6a8fb15d11d5 node DatanodeRegistration(127.0.0.1:33893, datanodeUuid=79a7a878-9292-4e46-8ee3-6d130a7b9ee3, infoPort=55167, infoSecurePort=0, ipcPort=56591, storageInfo=lv=-56;cid=testClusterID;nsid=446232316;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1231172 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[3A2118BED78F59EA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46540
[junit4] 2> 1231369 INFO (IPC Server handler 6 on 37863) [ ] BlockStateChange BLOCK* processReport: from storage DS-cd7bdb71-b2fa-4229-b459-574f40cd9d0f node DatanodeRegistration(127.0.0.1:47750, datanodeUuid=67b1f67c-059e-46b6-888b-167690304db2, infoPort=43383, infoSecurePort=0, ipcPort=56693, storageInfo=lv=-56;cid=testClusterID;nsid=446232316;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1231369 INFO (IPC Server handler 6 on 37863) [ ] BlockStateChange BLOCK* processReport: from storage DS-29adb3d7-dd15-4227-b0df-ed2c85a400e8 node DatanodeRegistration(127.0.0.1:47750, datanodeUuid=67b1f67c-059e-46b6-888b-167690304db2, infoPort=43383, infoSecurePort=0, ipcPort=56693, storageInfo=lv=-56;cid=testClusterID;nsid=446232316;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1231652 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1231654 INFO (Thread-49214) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1231654 INFO (Thread-49214) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1231752 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.ZkTestServer start zk server on port:60711
[junit4] 2> 1231767 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1231769 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1231771 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1231773 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1231774 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1231776 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1231777 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1231779 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1231780 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1231782 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1231783 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1231903 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/control-001/cores/collection1
[junit4] 2> 1231905 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1231906 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4203e973{/dr,null,AVAILABLE}
[junit4] 2> 1231907 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.ServerConnector Started ServerConnector@13968e10{HTTP/1.1,[http/1.1]}{127.0.0.1:35890}
[junit4] 2> 1231907 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.Server Started @1235412ms
[junit4] 2> 1231907 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/dr, hostPort=35890, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/control-001/cores}
[junit4] 2> 1231907 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1231908 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 1231908 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1231908 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1231908 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-29T18:11:05.058Z
[junit4] 2> 1231913 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1231913 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/control-001/solr.xml
[junit4] 2> 1231921 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1231921 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60711/solr
[junit4] 2> 1231951 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:35890_dr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35890_dr
[junit4] 2> 1231952 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:35890_dr ] o.a.s.c.Overseer Overseer (id=96846785073119236-127.0.0.1:35890_dr-n_0000000000) starting
[junit4] 2> 1231962 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:35890_dr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35890_dr
[junit4] 2> 1231966 INFO (OverseerStateUpdate-96846785073119236-127.0.0.1:35890_dr-n_0000000000) [n:127.0.0.1:35890_dr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1231989 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:35890_dr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/control-001/cores
[junit4] 2> 1231989 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:35890_dr ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1231993 INFO (OverseerStateUpdate-96846785073119236-127.0.0.1:35890_dr-n_0000000000) [n:127.0.0.1:35890_dr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1232006 WARN (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1232007 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1232021 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1232111 WARN (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1232113 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1232133 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1232148 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37863/solr_hdfs_home
[junit4] 2> 1232148 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1232149 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/control-001/cores/collection1], dataDir=[hdfs://localhost:37863/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 1232149 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e866f61
[junit4] 2> 1232150 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1232170 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1232170 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1232170 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1233945 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1233953 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1233987 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1233995 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1233995 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1233995 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1234015 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1234016 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.126953125, noCFSRatio=1.0]
[junit4] 2> 1234056 INFO (IPC Server handler 8 on 37863) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47750 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ef2addb3-abd7-4c8f-952b-6a8fb15d11d5:NORMAL:127.0.0.1:33893|RBW], ReplicaUC[[DISK]DS-29adb3d7-dd15-4227-b0df-ed2c85a400e8:NORMAL:127.0.0.1:47750|FINALIZED]]} size 0
[junit4] 2> 1234061 INFO (IPC Server handler 1 on 37863) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33893 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ef2addb3-abd7-4c8f-952b-6a8fb15d11d5:NORMAL:127.0.0.1:33893|RBW], ReplicaUC[[DISK]DS-29adb3d7-dd15-4227-b0df-ed2c85a400e8:NORMAL:127.0.0.1:47750|FINALIZED]]} size 0
[junit4] 2> 1234095 WARN (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1234152 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1234152 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1234152 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1234171 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1234171 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1234173 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=83.1455078125, floorSegmentMB=0.525390625, forceMergeDeletesPctAllowed=6.264120001581821, segmentsPerTier=18.0, maxCFSSegmentSizeMB=0.49609375, noCFSRatio=1.0
[junit4] 2> 1234289 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5cba00ca[collection1] main]
[junit4] 2> 1234291 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1234292 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1234301 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1234303 INFO (searcherExecutor-4087-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5cba00ca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1234303 INFO (coreLoadExecutor-4086-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549548563939196928
[junit4] 2> 1234316 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1234316 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1234317 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35890/dr/collection1/
[junit4] 2> 1234317 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1234317 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35890/dr/collection1/ has no replicas
[junit4] 2> 1234322 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35890/dr/collection1/ shard1
[junit4] 2> 1234473 INFO (coreZkRegister-4079-thread-1-processing-n:127.0.0.1:35890_dr x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35890_dr c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1234507 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1234509 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1234698 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001/cores/collection1
[junit4] 2> 1234699 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001
[junit4] 2> 1234699 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1234702 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@782851a0{/dr,null,AVAILABLE}
[junit4] 2> 1234702 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.ServerConnector Started ServerConnector@6b196e4e{HTTP/1.1,[http/1.1]}{127.0.0.1:40617}
[junit4] 2> 1234702 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.Server Started @1238208ms
[junit4] 2> 1234703 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/dr, hostPort=40617, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001/cores}
[junit4] 2> 1234708 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1234708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 1234708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1234708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1234708 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-29T18:11:07.858Z
[junit4] 2> 1234724 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1234724 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001/solr.xml
[junit4] 2> 1234733 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1234735 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60711/solr
[junit4] 2> 1234754 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:40617_dr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1234762 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:40617_dr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40617_dr
[junit4] 2> 1234764 INFO (zkCallback-1166-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1234765 INFO (zkCallback-1162-thread-1-processing-n:127.0.0.1:35890_dr) [n:127.0.0.1:35890_dr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1234765 INFO (zkCallback-1171-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1234782 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:40617_dr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001/cores
[junit4] 2> 1234782 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [n:127.0.0.1:40617_dr ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1234791 INFO (OverseerStateUpdate-96846785073119236-127.0.0.1:35890_dr-n_0000000000) [n:127.0.0.1:35890_dr ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1235803 WARN (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1235804 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1235820 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1235911 WARN (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1235913 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1235931 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1235932 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37863/solr_hdfs_home
[junit4] 2> 1235932 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1235933 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:37863/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 1235933 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3e866f61
[junit4] 2> 1235933 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 1235941 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1235941 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1235941 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1235952 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1235954 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 1235978 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37863/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 1235987 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1235987 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1235987 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1235994 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1235994 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.126953125, noCFSRatio=1.0]
[junit4] 2> 1236054 INFO (IPC Server handler 0 on 37863) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33893 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-cd7bdb71-b2fa-4229-b459-574f40cd9d0f:NORMAL:127.0.0.1:47750|RBW], ReplicaUC[[DISK]DS-f33145a3-a520-4056-99b8-11ec8d8416c5:NORMAL:127.0.0.1:33893|FINALIZED]]} size 0
[junit4] 2> 1236058 INFO (IPC Server handler 4 on 37863) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:47750 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-cd7bdb71-b2fa-4229-b459-574f40cd9d0f:NORMAL:127.0.0.1:47750|RBW], ReplicaUC[[DISK]DS-f33145a3-a520-4056-99b8-11ec8d8416c5:NORMAL:127.0.0.1:33893|FINALIZED]]} size 0
[junit4] 2> 1236077 WARN (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1236169 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1236169 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1236169 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1236188 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1236188 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1236203 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=83.1455078125, floorSegmentMB=0.525390625, forceMergeDeletesPctAllowed=6.264120001581821, segmentsPerTier=18.0, maxCFSSegmentSizeMB=0.49609375, noCFSRatio=1.0
[junit4] 2> 1236217 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@476497e2[collection1] main]
[junit4] 2> 1236218 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1236218 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1236218 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1236219 INFO (searcherExecutor-4098-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@476497e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1236220 INFO (coreLoadExecutor-4097-thread-1-processing-n:127.0.0.1:40617_dr) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549548565949317120
[junit4] 2> 1236228 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1236228 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1236228 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40617/dr/collection1/
[junit4] 2> 1236228 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1236228 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40617/dr/collection1/ has no replicas
[junit4] 2> 1236234 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40617/dr/collection1/ shard2
[junit4] 2> 1236345 INFO (coreZkRegister-4092-thread-1-processing-n:127.0.0.1:40617_dr x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:40617_dr c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1236411 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-2-001/cores/collection1
[junit4] 2> 1236412 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_3A2118BED78F59EA-001/shard-2-001
[junit4] 2> 1236413 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1236418 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[3A2118BED78F59EA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4fdd4462{/dr,null,AVAILABLE}
[junit4] 2> 123
[...truncated too long message...]
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.teardownClass(HdfsSyncSliceTest.java:46)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 4452132 INFO (SUITE-HdfsSyncSliceTest-seed#[3A2118BED78F59EA]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsSyncSliceTest_3A2118BED78F59EA-001
[junit4] 2> Oct 29, 2016 7:04:45 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1125, maxMBSortInHeap=7.543626528870972, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=sv, timezone=Indian/Mayotte
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=247502912,total=522715136
[junit4] 2> NOTE: All tests run in this JVM: [CollectionsAPISolrJTest, DebugComponentTest, TestDFRSimilarityFactory, TestSolrCLIRunExample, DistanceUnitsTest, TestStressReorder, AssignTest, TestReload, FieldMutatingUpdateProcessorTest, SolrPluginUtilsTest, BlockJoinFacetSimpleTest, SimpleFacetsTest, TestCollapseQParserPlugin, FullSolrCloudDistribCmdsTest, SimpleCollectionCreateDeleteTest, TestHashQParserPlugin, PeerSyncReplicationTest, ShowFileRequestHandlerTest, TestIntervalFaceting, OpenExchangeRatesOrgProviderTest, RequestHandlersTest, TestDocTermOrds, HdfsBasicDistributedZk2Test, ExplicitHLLTest, SolrXmlInZkTest, TestZkChroot, LukeRequestHandlerTest, StatsComponentTest, TestNamedUpdateProcessors, PrimUtilsTest, TestNonDefinedSimilarityFactory, TestSolrCloudWithDelegationTokens, SecurityConfHandlerTest, CdcrBootstrapTest, TestXmlQParserPlugin, TestDynamicLoading, TestManagedSynonymFilterFactory, CleanupOldIndexTest, TestPKIAuthenticationPlugin, TestLocalFSCloudBackupRestore, PrimitiveFieldTypeTest, RecoveryZkTest, TestConfigSets, TestExactSharedStatsCache, AddSchemaFieldsUpdateProcessorFactoryTest, SolrCoreCheckLockOnStartupTest, SpatialHeatmapFacetsTest, TestRecovery, SortSpecParsingTest, TestJmxMonitoredMap, ResponseHeaderTest, TestHighFrequencyDictionaryFactory, OverriddenZkACLAndCredentialsProvidersTest, RequestLoggingTest, BasicDistributedZk2Test, TestSSLRandomization, TestSchemaVersionResource, TestPhraseSuggestions, TestDynamicFieldCollectionResource, TestManagedResourceStorage, TestCloudInspectUtil, TestLeaderElectionWithEmptyReplica, TestSolrConfigHandlerCloud, TestBlendedInfixSuggestions, TestMinMaxOnMultiValuedField, NoCacheHeaderTest, AutoCommitTest, BadIndexSchemaTest, SuggesterWFSTTest, DistributedFacetPivotSmallTest, HdfsUnloadDistributedZkTest, UpdateParamsTest, ConnectionReuseTest, TestStressLiveNodes, TestSchemaNameResource, CircularListTest, DataDrivenBlockJoinTest, CloneFieldUpdateProcessorFactoryTest, CoreAdminRequestStatusTest, TestCollationField, TestRawResponseWriter, PluginInfoTest, TestReRankQParserPlugin, BigEndianAscendingWordDeserializerTest, ReplaceNodeTest, TestCloudManagedSchema, TestMergePolicyConfig, DirectUpdateHandlerOptimizeTest, CurrencyFieldXmlFileTest, JSONWriterTest, TestUtils, TermVectorComponentDistributedTest, BinaryUpdateRequestHandlerTest, DistributedQueryComponentOptimizationTest, TestPostingsSolrHighlighter, TestDownShardTolerantSearch, QueryParsingTest, TestSolrJ, TestLFUCache, BJQParserTest, OverseerTest, BasicDistributedZkTest, CollectionsAPIDistributedZkTest, ShardRoutingTest, TestReplicationHandler, TestCoreContainer, TestSolr4Spatial, SolrCoreTest, TestGroupingSearch, QueryElevationComponentTest, PeerSyncTest, ConvertedLegacyTest, TestSort, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, TestIndexSearcher, SoftAutoCommitTest, CurrencyFieldOpenExchangeTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, SpatialFilterTest, TestCSVLoader, PolyFieldTest, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, TestWriterPerf, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, TestSolrQueryParser, RegexBoostProcessorTest, IndexSchemaTest, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, PingRequestHandlerTest, CSVRequestHandlerTest, HighlighterConfigTest, SOLR749Test, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, BadComponentTest, TestDocSet, TestBinaryField, TestSearchPerf, MinimalSchemaTest, TestConfig, OutputWriterTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, DateMathParserTest, ChaosMonkeyNothingIsSafeTest, CdcrRequestHandlerTest, CdcrVersionReplicationTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, DeleteReplicaTest, DistribJoinFromCollectionTest, DistributedQueueTest, DistributedVersionInfoTest, HttpPartitionTest, MigrateRouteKeyTest, OverseerModifyCollectionTest, OverseerStatusTest, ReplicationFactorTest, RestartWhileUpdatingTest, SSLMigrationTest, ShardSplitTest, SolrCloudExampleTest, TestCloudRecovery, TestClusterProperties, TestCollectionAPI, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestExclusionRuleCollectionAccess, TestLockTree, TestMiniSolrCloudClusterSSL, TestOnReconnectListenerSupport, TestRandomFlRTGCloud, TriLevelCompositeIdRoutingTest, HdfsBasicDistributedZkTest, HdfsRecoverLeaseTest, HdfsSyncSliceTest]
[junit4] Completed [519/648 (3!)] on J2 in 77.08s, 1 test, 1 failure <<< FAILURES!
[...truncated 60767 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 187 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/187/
4 tests failed.
FAILED: org.apache.lucene.spatial.geopoint.search.TestLegacyGeoPointQuery.testRandomBig
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([2FDE41DE24EAFBFD:A8893C51B5B3877D]:0)
at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:129)
at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:35)
at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1088)
at org.apache.lucene.util.fst.FST.pack(FST.java:1769)
at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:164)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
at org.apache.lucene.geo.BaseGeoPointTestCase.verifyRandomRectangles(BaseGeoPointTestCase.java:774)
at org.apache.lucene.geo.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:743)
at org.apache.lucene.geo.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:692)
at org.apache.lucene.geo.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:623)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop
Error Message:
Timeout while trying to assert number of documents @ target_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ target_collection
at __randomizedtesting.SeedInfo.seed([16E038A66AE0973A:9523E6357E381CB3]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationStartStop(CdcrReplicationDistributedZkTest.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: expected:<10> but was:<6>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testSpecificConfigsets
Error Message:
KeeperErrorCode = NoNode for /collections/withconfigset2
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/withconfigset2
at __randomizedtesting.SeedInfo.seed([16E038A66AE0973A:3B9E77FC9DC42D36]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:356)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:353)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:353)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testSpecificConfigsets(CollectionsAPIDistributedZkTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Expected to see collection awhollynewcollection_0 null Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={ "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected to see collection awhollynewcollection_0
null
Last available state: DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/3)={
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([16E038A66AE0973A:5E954C126CD3B8AF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:236)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9306 lines...]
[junit4] Suite: org.apache.lucene.spatial.geopoint.search.TestLegacyGeoPointQuery
[junit4] IGNOR/A 0.02s J1 | TestLegacyGeoPointQuery.testRandomDistance
[junit4] > Assumption #1: legacy encoding is too slow/hangs on this test
[junit4] IGNOR/A 0.01s J1 | TestLegacyGeoPointQuery.testSamePointManyTimes
[junit4] > Assumption #1: legacy encoding goes OOM on this test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLegacyGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=2FDE41DE24EAFBFD -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=hi-IN -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 534s J1 | TestLegacyGeoPointQuery.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] > at __randomizedtesting.SeedInfo.seed([2FDE41DE24EAFBFD:A8893C51B5B3877D]:0)
[junit4] > at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:129)
[junit4] > at org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.java:35)
[junit4] > at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:1088)
[junit4] > at org.apache.lucene.util.fst.FST.pack(FST.java:1769)
[junit4] > at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
[junit4] > at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:267)
[junit4] > at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:401)
[junit4] > at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:164)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4312)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3889)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2052)
[junit4] > at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4953)
[junit4] > at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4991)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:4982)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1565)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1307)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.verifyRandomRectangles(BaseGeoPointTestCase.java:774)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:743)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:692)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:623)
[junit4] IGNOR/A 0.00s J1 | TestLegacyGeoPointQuery.testRandomDistanceHuge
[junit4] > Assumption #1: legacy encoding is too slow/hangs on this test
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/spatial/test/J1/temp/lucene.spatial.geopoint.search.TestLegacyGeoPointQuery_2FDE41DE24EAFBFD-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {field=PostingsFormat(name=LuceneFixedGap), string=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=Lucene50(blocksize=128), point=PostingsFormat(name=Memory doPackFST= true)}, docValues:{field=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene54), point=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=526, maxMBSortInHeap=7.908267081728155, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=hi-IN, timezone=Africa/Porto-Novo
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=250342448,total=393740288
[junit4] 2> NOTE: All tests run in this JVM: [TestLegacyGeoPointQuery]
[junit4] Completed [4/4 (1!)] on J1 in 736.97s, 40 tests, 1 error, 3 skipped <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/spatial/test/temp/junit4-J1-20161028_184355_182.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/heapdumps/java_pid6182.hprof ...
[junit4] Heap dump file created [438570291 bytes in 0.907 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 3203 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/init-core-data-001
[junit4] 2> 1327239 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[16E038A66AE0973A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1327240 INFO (SUITE-CdcrReplicationDistributedZkTest-seed#[16E038A66AE0973A]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jcaxe/
[junit4] 2> 1327243 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1327244 INFO (Thread-50686) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1327244 INFO (Thread-50686) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1327344 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkTestServer start zk server on port:38444
[junit4] 2> 1327357 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1327360 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1327362 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1327364 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1327366 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1327367 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1327370 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1327373 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1327374 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1327376 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1327378 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1327383 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testUpdateLogSynchronisation
[junit4] 2> 1327517 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/cores/collection1
[junit4] 2> 1327519 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1327521 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c602b15{/jcaxe,null,AVAILABLE}
[junit4] 2> 1327525 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.ServerConnector Started ServerConnector@5b45c43e{SSL,[ssl, http/1.1]}{127.0.0.1:42909}
[junit4] 2> 1327526 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server Started @1331099ms
[junit4] 2> 1327526 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/jcaxe, hostPort=42909, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/cores}
[junit4] 2> 1327526 ERROR (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1327526 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1327526 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1327526 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1327527 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-28T20:02:42.425Z
[junit4] 2> 1327530 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1327530 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/solr.xml
[junit4] 2> 1327537 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1327538 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38444/solr
[junit4] 2> 1327568 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42909_jcaxe
[junit4] 2> 1327569 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.Overseer Overseer (id=96841561679003653-127.0.0.1:42909_jcaxe-n_0000000000) starting
[junit4] 2> 1327578 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42909_jcaxe
[junit4] 2> 1327583 INFO (zkCallback-1300-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1327591 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/cores
[junit4] 2> 1327591 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1327596 INFO (OverseerStateUpdate-96841561679003653-127.0.0.1:42909_jcaxe-n_0000000000) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1328598 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.c.ZkController Only one config set found in zk - using it:conf1
[junit4] 2> 1328610 WARN (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1328611 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1328623 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1328706 WARN (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1328708 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1328728 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 1328729 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-001/cores/collection1/data/]
[junit4] 2> 1328729 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3b3cf9
[junit4] 2> 1328730 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6822be7b
[junit4] 2> 1328743 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1328743 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1328744 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1328744 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1328744 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1328744 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@20d73eb8[collection1] main]
[junit4] 2> 1328745 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1328746 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1328746 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1328749 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 1328750 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
[junit4] 2> 1328752 INFO (searcherExecutor-4164-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@20d73eb8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1328752 INFO (coreLoadExecutor-4163-thread-1-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549464988444262400
[junit4] 2> 1328761 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1328761 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1328761 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42909/jcaxe/collection1/
[junit4] 2> 1328761 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1328761 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:42909/jcaxe/collection1/ has no replicas
[junit4] 2> 1328765 INFO (zkCallback-1300-thread-2-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard2
[junit4] 2> 1328766 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42909/jcaxe/collection1/ shard2
[junit4] 2> 1328917 INFO (coreZkRegister-4156-thread-1-processing-n:127.0.0.1:42909_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:42909_jcaxe c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1329222 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/cores/collection1
[junit4] 2> 1329224 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1329226 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5656a8b1{/jcaxe,null,AVAILABLE}
[junit4] 2> 1329228 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.ServerConnector Started ServerConnector@6b468ccb{SSL,[ssl, http/1.1]}{127.0.0.1:55296}
[junit4] 2> 1329229 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server Started @1332802ms
[junit4] 2> 1329229 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/jcaxe, hostPort=55296, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/cores}
[junit4] 2> 1329229 ERROR (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1329229 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1329229 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1329229 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1329230 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-28T20:02:44.128Z
[junit4] 2> 1329233 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1329233 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/solr.xml
[junit4] 2> 1329240 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1329241 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38444/solr
[junit4] 2> 1329253 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1329258 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55296_jcaxe
[junit4] 2> 1329261 INFO (zkCallback-1300-thread-3-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1329262 INFO (zkCallback-1306-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1329283 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/cores
[junit4] 2> 1329283 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1329286 INFO (OverseerStateUpdate-96841561679003653-127.0.0.1:42909_jcaxe-n_0000000000) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1330298 WARN (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1330298 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1330311 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1330394 WARN (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1330395 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1330414 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 1330415 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-002/cores/collection1/data/]
[junit4] 2> 1330415 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3b3cf9
[junit4] 2> 1330416 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@331c3a94
[junit4] 2> 1330431 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1330431 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1330432 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1330432 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1330433 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1330433 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@96cadfb[collection1] main]
[junit4] 2> 1330434 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1330435 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1330435 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1330438 INFO (searcherExecutor-4176-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@96cadfb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1330438 INFO (coreLoadExecutor-4175-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549464990212161536
[junit4] 2> 1330447 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1330447 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1330447 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:55296/jcaxe/collection1/
[junit4] 2> 1330447 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1330447 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:55296/jcaxe/collection1/ has no replicas
[junit4] 2> 1330453 INFO (zkCallback-1306-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard1
[junit4] 2> 1330455 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:55296/jcaxe/collection1/ shard1
[junit4] 2> 1330607 INFO (coreZkRegister-4170-thread-1-processing-n:127.0.0.1:55296_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node2) [n:127.0.0.1:55296_jcaxe c:tmp_collection s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1330918 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/cores/collection1
[junit4] 2> 1330921 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1330923 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d9bb13{/jcaxe,null,AVAILABLE}
[junit4] 2> 1330925 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.ServerConnector Started ServerConnector@7f07c27b{SSL,[ssl, http/1.1]}{127.0.0.1:55075}
[junit4] 2> 1330925 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server Started @1334499ms
[junit4] 2> 1330925 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard3, hostContext=/jcaxe, hostPort=55075, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/cores}
[junit4] 2> 1330926 ERROR (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1330926 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1330926 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1330926 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1330926 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-28T20:02:45.824Z
[junit4] 2> 1330930 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1330930 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/solr.xml
[junit4] 2> 1330937 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1330938 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38444/solr
[junit4] 2> 1330950 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1330957 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55075_jcaxe
[junit4] 2> 1330959 INFO (zkCallback-1306-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1330960 INFO (zkCallback-1312-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1330959 INFO (zkCallback-1300-thread-2-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1330970 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/cores
[junit4] 2> 1330970 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1330972 INFO (OverseerStateUpdate-96841561679003653-127.0.0.1:42909_jcaxe-n_0000000000) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1331985 WARN (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1331985 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1331998 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1332085 WARN (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1332087 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1332106 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 1332106 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-003/cores/collection1/data/]
[junit4] 2> 1332106 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3b3cf9
[junit4] 2> 1332108 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@cbebe77
[junit4] 2> 1332123 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1332123 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1332124 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1332124 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1332125 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1332125 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e792f0c[collection1] main]
[junit4] 2> 1332127 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1332127 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1332127 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1332130 INFO (searcherExecutor-4188-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5e792f0c[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1332130 INFO (coreLoadExecutor-4187-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549464991986352128
[junit4] 2> 1332135 INFO (coreZkRegister-4182-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1332135 INFO (updateExecutor-1309-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1332136 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1332136 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1332136 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1332136 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1332136 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:42909/jcaxe/collection1/] and I am [https://127.0.0.1:55075/jcaxe/collection1/]
[junit4] 2> 1332139 INFO (recoveryExecutor-1310-thread-1-processing-n:127.0.0.1:55075_jcaxe x:collection1 s:shard2 c:tmp_collection r:core_node3) [n:127.0.0.1:55075_jcaxe c:tmp_collection s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:42909/jcaxe]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:55075_jcaxe&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1332202 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 1332203 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of tmp_collection) have state: recovering
[junit4] 2> 1332203 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:55075_jcaxe, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"https://127.0.0.1:55075/jcaxe","node_name":"127.0.0.1:55075_jcaxe","state":"down"}
[junit4] 2> 1332599 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/cores/collection1
[junit4] 2> 1332602 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1332603 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6bb7d33a{/jcaxe,null,AVAILABLE}
[junit4] 2> 1332605 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.ServerConnector Started ServerConnector@639b64ab{SSL,[ssl, http/1.1]}{127.0.0.1:36430}
[junit4] 2> 1332607 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.e.j.s.Server Started @1336180ms
[junit4] 2> 1332607 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard4, hostContext=/jcaxe, hostPort=36430, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/cores}
[junit4] 2> 1332607 ERROR (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1332608 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1332608 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1332608 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1332608 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-28T20:02:47.506Z
[junit4] 2> 1332611 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1332611 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/solr.xml
[junit4] 2> 1332618 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1332619 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38444/solr
[junit4] 2> 1332630 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:36430_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1332636 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:36430_jcaxe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36430_jcaxe
[junit4] 2> 1332638 INFO (zkCallback-1306-thread-1-processing-n:127.0.0.1:55296_jcaxe) [n:127.0.0.1:55296_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1332639 INFO (zkCallback-1312-thread-1-processing-n:127.0.0.1:55075_jcaxe) [n:127.0.0.1:55075_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1332639 INFO (zkCallback-1300-thread-3-processing-n:127.0.0.1:42909_jcaxe) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1332640 INFO (zkCallback-1319-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1332657 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:36430_jcaxe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/cores
[junit4] 2> 1332657 INFO (TEST-CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation-seed#[16E038A66AE0973A]) [n:127.0.0.1:36430_jcaxe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1332659 INFO (OverseerStateUpdate-96841561679003653-127.0.0.1:42909_jcaxe-n_0000000000) [n:127.0.0.1:42909_jcaxe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1333203 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:55075_jcaxe, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"https://127.0.0.1:55075/jcaxe","node_name":"127.0.0.1:55075_jcaxe","state":"recovering"}
[junit4] 2> 1333203 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 1333203 INFO (qtp662934893-56931) [n:127.0.0.1:42909_jcaxe ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:55075_jcaxe&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 1333676 WARN (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1333677 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1333689 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1333775 WARN (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1333776 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1333795 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
[junit4] 2> 1333796 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.CdcrReplicationDistributedZkTest_16E038A66AE0973A-001/jetty-004/cores/collection1/data/]
[junit4] 2> 1333796 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e3b3cf9
[junit4] 2> 1333797 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2b27024c
[junit4] 2> 1333812 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1333812 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1333815 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1333815 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1333816 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1333817 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@79f623cf[collection1] main]
[junit4] 2> 1333818 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1333819 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1333819 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1333821 INFO (searcherExecutor-4199-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79f623cf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1333821 INFO (coreLoadExecutor-4198-thread-1-processing-n:127.0.0.1:36430_jcaxe) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549464993759494144
[junit4] 2> 1333826 INFO (coreZkRegister-4193-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1333826 INFO (updateExecutor-1316-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1333827 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1333827 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1333828 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1333828 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1333828 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:55296/jcaxe/collection1/] and I am [https://127.0.0.1:36430/jcaxe/collection1/]
[junit4] 2> 1333831 INFO (recoveryExecutor-1317-thread-1-processing-n:127.0.0.1:36430_jcaxe x:collection1 s:shard1 c:tmp_collection r:core_node4) [n:127.0.0.1:36430_jcaxe c:tmp_collection s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:55296/jcaxe]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:36430_jcaxe&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1333891 INFO (qtp2084451162-56964) [n:127.0.0.1:55296_jcaxe ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 1333891 INFO (qtp2084451162-56964) [n:127.0.0.1:55296_jcaxe ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of tmp_collection) have state: recovering
[junit4] 2> 1333892 INFO (qtp2084451162-56964) [n:127.0.0.1:55296_jcaxe ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:36430_jcaxe, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1","base_url":"https://127.0.0.1:36430/jcaxe","node_name":"127.0.0.1:36430_jcaxe","state":"down"}
[junit4] 1> / (2)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /zookeeper (1)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /solr (9)
[junit4] 1> /solr/configs (1)
[junit4] 1> /solr/configs/conf1 (11)
[junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/enumsConfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4] 1> DATA: ...supressed...
[...truncated too long message...]
.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:198)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:52)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 4272430 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[16E038A66AE0973A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37128 37128
[junit4] 2> 4272430 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[16E038A66AE0973A]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_16E038A66AE0973A-001
[junit4] 2> Oct 28, 2016 8:51:47 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=ar-DZ, timezone=Asia/Macao
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=78504840,total=520093696
[junit4] 2> NOTE: All tests run in this JVM: [TestCorePropertiesReload, TestManagedSchema, FullHLLTest, ZkStateReaderTest, TestStressLucene, TestFoldingMultitermQuery, TestJmxIntegration, TestCopyFieldCollectionResource, DistributedSuggestComponentTest, TestValueSourceCache, TestDistributedSearch, RuleEngineTest, MultiThreadedOCPTest, TestEmbeddedSolrServerConstructors, TestInitQParser, ChaosMonkeyNothingIsSafeTest, TestSolrConfigHandler, SegmentsInfoRequestHandlerTest, TestSchemaResource, TestSchemaManager, CurrencyFieldOpenExchangeTest, TestDocSet, TestPerFieldSimilarityClassic, TestJsonRequest, TestExactStatsCache, TestNumericTerms64, TestOmitPositions, TestRuleBasedAuthorizationPlugin, DistribDocExpirationUpdateProcessorTest, TestSolrConfigHandlerConcurrent, QueryElevationComponentTest, CacheHeaderTest, MBeansHandlerTest, TestRestManager, DateRangeFieldTest, TestOrdValues, TestFilteredDocIdSet, TestBadConfig, TolerantUpdateProcessorTest, TestRandomDVFaceting, TestSolr4Spatial, PeerSyncTest, HdfsDirectoryTest, TestSerializedLuceneMatchVersion, NumericFieldsTest, TriLevelCompositeIdRoutingTest, TestBinaryResponseWriter, TestChildDocTransformer, TestJsonFacetRefinement, SolrCLIZkUtilsTest, TestFieldCacheSort, TestDeleteCollectionOnDownNodes, CloudExitableDirectoryReaderTest, NotRequiredUniqueKeyTest, TestMiniSolrCloudClusterSSL, HdfsRecoveryZkTest, PeerSyncWithIndexFingerprintCachingTest, TestExpandComponent, DistributedExpandComponentTest, BasicZkTest, CdcrVersionReplicationTest, FileBasedSpellCheckerTest, StandardRequestHandlerTest, TestComplexPhraseQParserPlugin, TestRandomCollapseQParserPlugin, TestFunctionQuery, DocExpirationUpdateProcessorFactoryTest, RequiredFieldsTest, PreAnalyzedUpdateProcessorTest, TestClusterProperties, TestSQLHandler, TestRequestForwarding, AnalysisAfterCoreReloadTest, TestLeaderElectionZkExpiry, TestNRTOpen, CoreMergeIndexesAdminHandlerTest, TestTrieFacet, TestAtomicUpdateErrorCases, TestDefaultStatsCache, OverseerStatusTest, ResourceLoaderTest, TestImplicitCoreProperties, ZkNodePropsTest, ConcurrentDeleteAndCreateCollectionTest, CdcrReplicationDistributedZkTest, TestCrossCoreJoin, TestJoin, TestPseudoReturnFields, TestRealTimeGet, TestReplicationHandler, FastVectorHighlighterTest, TestNoOpRegenerator, DistributedSpellCheckComponentTest, QueryEqualityTest, TestSolrCoreProperties, SynonymTokenizerTest, QueryResultKeyTest, BlockJoinFacetDistribTest, TestLockTree, TestDocumentBuilder, ScriptEngineTest, DistributedFacetPivotLongTailTest, TestStressRecovery, HdfsWriteToMultipleCollectionsTest, DistributedVersionInfoTest, HdfsChaosMonkeyNothingIsSafeTest, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [510/648 (2!)] on J0 in 255.52s, 20 tests, 1 failure, 1 error <<< FAILURES!
[...truncated 11482 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:717: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid6182.hprof
Total time: 261 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 186 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/186/
3 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=11454, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11454, name=collection4, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:45246
at __randomizedtesting.SeedInfo.seed([78DB05D1F24CA4A1]:0)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:45246
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 11 more
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1477573304251,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1477573304251,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1477573304251,generation=3,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _7.cfe, _7.cfs, _7.si, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1477573304251,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}]> but was:<[{indexVersion=1477573304251,generation=2,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, _6.cfe, _6.cfs, _6.si, segments_2]}, {indexVersion=1477573304251,generation=3,filelist=[_3.fdt, _3.fdx, _3.fnm, _3.nvd, _3.nvm, _3.si, _3_LuceneFixedGap_0.doc, _3_LuceneFixedGap_0.tib, _3_LuceneFixedGap_0.tii, _7.cfe, _7.cfs, _7.si, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([78DB05D1F24CA4A1:5D0C1EE18204AAA2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1229)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=8472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=8472, name=collection0, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57028
at __randomizedtesting.SeedInfo.seed([78DB05D1F24CA4A1]:0)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:57028
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 11 more
Build Log:
[...truncated 11420 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/init-core-data-001
[junit4] 2> 284236 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[78DB05D1F24CA4A1]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 284236 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[78DB05D1F24CA4A1]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 284242 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 284256 INFO (Thread-6504) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 284256 INFO (Thread-6504) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 284356 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkTestServer start zk server on port:54260
[junit4] 2> 284557 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 284560 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 284562 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 284563 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 284565 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 284566 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 284567 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 284568 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 284570 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 284571 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 284572 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 284686 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/cores/collection1
[junit4] 2> 284688 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 284689 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74409b0b{/,null,AVAILABLE}
[junit4] 2> 284691 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Started ServerConnector@246af9d6{HTTP/1.1,[http/1.1]}{127.0.0.1:40666}
[junit4] 2> 284691 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server Started @287755ms
[junit4] 2> 284691 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/tempDir-001/control/data, hostContext=/, hostPort=40666, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/cores}
[junit4] 2> 284692 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 284693 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 284698 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 284698 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 284698 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-27T12:25:13.713Z
[junit4] 2> 284703 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 284703 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/solr.xml
[junit4] 2> 284712 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 284713 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54260/solr
[junit4] 2> 284753 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:40666_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40666_
[junit4] 2> 284754 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:40666_ ] o.a.s.c.Overseer Overseer (id=96834100479393796-127.0.0.1:40666_-n_0000000000) starting
[junit4] 2> 284766 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:40666_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40666_
[junit4] 2> 284768 INFO (zkCallback-372-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 284790 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:40666_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/cores
[junit4] 2> 284790 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:40666_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 284794 INFO (OverseerStateUpdate-96834100479393796-127.0.0.1:40666_-n_0000000000) [n:127.0.0.1:40666_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 284812 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 284826 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 284917 WARN (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 284919 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 284932 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 284933 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/control-001/cores/collection1/data/]
[junit4] 2> 284933 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177a275e
[junit4] 2> 284934 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@55efb9ab
[junit4] 2> 284938 WARN (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 284947 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 284947 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 284947 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 284947 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 284948 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 284950 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@29f6a17[collection1] main]
[junit4] 2> 284951 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 284952 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 284952 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 284952 INFO (searcherExecutor-1097-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@29f6a17[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 284952 INFO (coreLoadExecutor-1096-thread-1-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549345608399060992
[junit4] 2> 284968 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 284968 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 284968 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40666/collection1/
[junit4] 2> 284968 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 284968 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40666/collection1/ has no replicas
[junit4] 2> 284981 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40666/collection1/ shard1
[junit4] 2> 285132 INFO (coreZkRegister-1089-thread-1-processing-n:127.0.0.1:40666_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:40666_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 285297 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 285298 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 285298 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 285416 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/cores/collection1
[junit4] 2> 285416 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001
[junit4] 2> 285417 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 285420 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a42a95d{/,null,AVAILABLE}
[junit4] 2> 285420 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Started ServerConnector@375ba0cd{HTTP/1.1,[http/1.1]}{127.0.0.1:47666}
[junit4] 2> 285420 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server Started @288484ms
[junit4] 2> 285420 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=47666, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/cores}
[junit4] 2> 285421 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 285422 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 285422 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285422 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285422 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-27T12:25:14.437Z
[junit4] 2> 285425 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 285425 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/solr.xml
[junit4] 2> 285432 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 285432 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54260/solr
[junit4] 2> 285445 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 285453 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47666_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47666_
[junit4] 2> 285455 INFO (zkCallback-376-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 285455 INFO (zkCallback-372-thread-2-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 285455 INFO (zkCallback-382-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 285465 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47666_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/cores
[junit4] 2> 285466 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47666_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 285468 INFO (OverseerStateUpdate-96834100479393796-127.0.0.1:40666_-n_0000000000) [n:127.0.0.1:40666_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 285572 INFO (zkCallback-382-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 286481 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 286495 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 286594 WARN (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 286597 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 286613 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 286613 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 286614 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177a275e
[junit4] 2> 286615 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@79861017
[junit4] 2> 286621 WARN (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 286632 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 286632 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 286633 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 286633 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 286633 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 286634 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@629037cd[collection1] main]
[junit4] 2> 286635 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 286636 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 286636 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 286637 INFO (searcherExecutor-1108-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@629037cd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 286637 INFO (coreLoadExecutor-1107-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549345610165911552
[junit4] 2> 286645 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 286645 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 286645 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:47666/collection1/
[junit4] 2> 286645 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 286645 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:47666/collection1/ has no replicas
[junit4] 2> 286650 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:47666/collection1/ shard2
[junit4] 2> 286754 INFO (zkCallback-382-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 286801 INFO (coreZkRegister-1102-thread-1-processing-n:127.0.0.1:47666_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:47666_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 286911 INFO (zkCallback-382-thread-1-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 287065 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/cores/collection1
[junit4] 2> 287066 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001
[junit4] 2> 287067 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 287068 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c49d173{/,null,AVAILABLE}
[junit4] 2> 287069 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Started ServerConnector@5375260b{HTTP/1.1,[http/1.1]}{127.0.0.1:43514}
[junit4] 2> 287069 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server Started @290133ms
[junit4] 2> 287069 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=43514, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/cores}
[junit4] 2> 287070 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 287070 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 287070 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 287070 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 287070 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-27T12:25:16.085Z
[junit4] 2> 287075 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 287075 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/solr.xml
[junit4] 2> 287081 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 287082 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54260/solr
[junit4] 2> 287096 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 287101 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:43514_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43514_
[junit4] 2> 287104 INFO (zkCallback-372-thread-4-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 287118 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 287121 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 287135 INFO (zkCallback-376-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 287139 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:43514_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/cores
[junit4] 2> 287139 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:43514_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 287150 INFO (OverseerStateUpdate-96834100479393796-127.0.0.1:40666_-n_0000000000) [n:127.0.0.1:40666_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 287254 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 287254 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 288163 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 288187 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 288278 WARN (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 288280 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 288294 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 288294 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 288294 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177a275e
[junit4] 2> 288296 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@e892078
[junit4] 2> 288301 WARN (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 288312 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 288312 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 288312 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 288312 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 288313 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 288314 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4e3c9a20[collection1] main]
[junit4] 2> 288318 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 288318 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 288318 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 288319 INFO (coreLoadExecutor-1118-thread-1-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549345611929616384
[junit4] 2> 288321 INFO (searcherExecutor-1119-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4e3c9a20[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 288335 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 288335 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 288335 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43514/collection1/
[junit4] 2> 288335 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 288335 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43514/collection1/ has no replicas
[junit4] 2> 288340 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43514/collection1/ shard1
[junit4] 2> 288443 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 288443 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 288491 INFO (coreZkRegister-1113-thread-1-processing-n:127.0.0.1:43514_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:43514_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 288598 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 288598 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 288758 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/cores/collection1
[junit4] 2> 288759 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001
[junit4] 2> 288761 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 288763 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@698b8aad{/,null,AVAILABLE}
[junit4] 2> 288763 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Started ServerConnector@5cd184bb{HTTP/1.1,[http/1.1]}{127.0.0.1:47006}
[junit4] 2> 288764 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server Started @291827ms
[junit4] 2> 288764 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=47006, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/cores}
[junit4] 2> 288764 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 288765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 288765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 288765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 288765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-27T12:25:17.780Z
[junit4] 2> 288776 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 288776 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/solr.xml
[junit4] 2> 288782 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 288783 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54260/solr
[junit4] 2> 288795 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 288802 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47006_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47006_
[junit4] 2> 288804 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 288804 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 288805 INFO (zkCallback-376-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 288805 INFO (zkCallback-372-thread-3-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 288806 INFO (zkCallback-394-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 288828 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47006_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/cores
[junit4] 2> 288828 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:47006_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 288831 INFO (OverseerStateUpdate-96834100479393796-127.0.0.1:40666_-n_0000000000) [n:127.0.0.1:40666_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 288934 INFO (zkCallback-394-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 288934 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 288934 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 289844 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 289865 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 289961 WARN (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 289963 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 289991 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 289991 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 289992 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@177a275e
[junit4] 2> 289993 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@47faa756
[junit4] 2> 289999 WARN (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 290010 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 290010 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 290011 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 290011 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 290015 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=32, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 290015 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@76680019[collection1] main]
[junit4] 2> 290016 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 290016 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 290017 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 290018 INFO (searcherExecutor-1130-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@76680019[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 290018 INFO (coreLoadExecutor-1129-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549345613711147008
[junit4] 2> 290033 INFO (coreZkRegister-1124-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 290042 INFO (updateExecutor-391-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 290053 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 290054 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 290054 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 290054 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 290054 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:47666/collection1/] and I am [http://127.0.0.1:47006/collection1/]
[junit4] 2> 290067 INFO (recoveryExecutor-392-thread-1-processing-n:127.0.0.1:47006_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:47006_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:47666]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:47006_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 290081 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 290081 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 290082 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:47006_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47006","node_name":"127.0.0.1:47006_","state":"down"}
[junit4] 2> 290160 INFO (zkCallback-394-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 290160 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 290160 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 290487 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-4-001/cores/collection1
[junit4] 2> 290487 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-4-001
[junit4] 2> 290488 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 290490 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a5f24a2{/,null,AVAILABLE}
[junit4] 2> 290491 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Started ServerConnector@41a65ba2{HTTP/1.1,[http/1.1]}{127.0.0.1:57028}
[junit4] 2> 290491 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.Server Started @293554ms
[junit4] 2> 290491 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=57028, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-4-001/cores}
[junit4] 2> 290491 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 290495 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 290495 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 290495 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 290495 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-27T12:25:19.510Z
[junit4] 2> 290505 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 290505 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-4-001/solr.xml
[junit4] 2> 290517 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 290518 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54260/solr
[junit4] 2> 290545 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:57028_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 290552 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:57028_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57028_
[junit4] 2> 290555 INFO (zkCallback-376-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290555 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290555 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290557 INFO (zkCallback-394-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290557 INFO (zkCallback-372-thread-3-processing-n:127.0.0.1:40666_) [n:127.0.0.1:40666_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290559 INFO (zkCallback-401-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 290576 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:57028_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_78DB05D1F24CA4A1-001/shard-4-001/cores
[junit4] 2> 290576 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[78DB05D1F24CA4A1]) [n:127.0.0.1:57028_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 290579 INFO (OverseerStateUpdate-96834100479393796-127.0.0.1:40666_-n_0000000000) [n:127.0.0.1:40666_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 290682 INFO (zkCallback-394-thread-1-processing-n:127.0.0.1:47006_) [n:127.0.0.1:47006_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 290682 INFO (zkCallback-401-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 290682 INFO (zkCallback-382-thread-2-processing-n:127.0.0.1:47666_) [n:127.0.0.1:47666_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 290683 INFO (zkCallback-388-thread-2-processing-n:127.0.0.1:43514_) [n:127.0.0.1:43514_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 291082 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:47006_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47006","node_name":"127.0.0.1:47006_","state":"recovering"}
[junit4] 2> 291082 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 291082 INFO (qtp245026796-8217) [n:127.0.0.1:47666_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:47006_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 291599 INFO (coreLoadExecutor-1140-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 291630 INFO (coreLoadExecutor-1140-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 291762 WARN (coreLoadExecutor-1140-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 291765 INFO (coreLoadExecutor-1140-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 291797 INFO (coreLoadExecutor-1140-thread-1-processing-n:127.0.0.1:57028_) [n:127.0.0.1:57028_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 29
[...truncated too long message...]
unit4] 2> 2795302 INFO (Thread-15860) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=12.8916015625, floorSegmentMB=0.6455078125, forceMergeDeletesPctAllowed=12.798059564856853, segmentsPerTier=11.0, maxCFSSegmentSizeMB=0.291015625, noCFSRatio=0.13102173578962503
[junit4] 2> 2795303 INFO (Thread-15860) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2795304 INFO (Thread-15860) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2f99408[collection1] main]
[junit4] 2> 2795304 INFO (searcherExecutor-10123-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2f99408[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_39(6.3.0):C39420) Uninverting(_3a(6.3.0):c459)))}
[junit4] 2> 2795304 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher removing old index directory MockDirectoryWrapper(RAMDirectory@7cfbc2e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@60031fa0)
[junit4] 2> 2795304 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=98.197265625, floorSegmentMB=0.564453125, forceMergeDeletesPctAllowed=1.163315092110938, segmentsPerTier=4.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2795305 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2795305 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@647f5528[collection1] main]
[junit4] 2> 2795305 INFO (searcherExecutor-10123-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@647f5528[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_39(6.3.0):C39420) Uninverting(_3a(6.3.0):c459)))}
[junit4] 2> 2795306 INFO (qtp1128341120-113016) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:57538/solr/collection1/replication&command=fetchindex} status=0 QTime=133
[junit4] 2> 2795306 INFO (qtp1128341120-113016) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@37453fd8
[junit4] 2> 2795309 INFO (qtp1128341120-113010) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=collection1&action=status&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2795322 INFO (qtp1128341120-113014) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=39879 status=0 QTime=11
[junit4] 2> 2795322 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 39879 docs
[junit4] 2> 2795324 INFO (qtp1665613473-113032) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2795326 INFO (qtp1665613473-113027) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2795328 INFO (qtp1128341120-113011) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2795329 INFO (qtp1665613473-113034) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2795331 INFO (qtp1128341120-113016) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2795331 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTestStressReplication
[junit4] 2> 2795332 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@55f1cba2{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2795332 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1222926890
[junit4] 2> 2795332 INFO (coreCloseExecutor-10127-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@153c484d
[junit4] 2> 2795334 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@717f4b4e{/solr,null,UNAVAILABLE}
[junit4] 2> 2795335 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@112721aa{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2795335 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1958583178
[junit4] 2> 2795336 INFO (coreCloseExecutor-10129-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4e7d9472
[junit4] 2> 2795337 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[78DB05D1F24CA4A1]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4b3ac9b6{/solr,null,UNAVAILABLE}
[junit4] 2> 2795339 INFO (SUITE-TestReplicationHandler-seed#[78DB05D1F24CA4A1]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_78DB05D1F24CA4A1-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {name=PostingsFormat(name=LuceneFixedGap), id=PostingsFormat(name=LuceneFixedGap), newname=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=900, maxMBSortInHeap=7.5218494121809325, sim=ClassicSimilarity, locale=es-UY, timezone=Africa/Asmera
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=188926968,total=525336576
[junit4] 2> NOTE: All tests run in this JVM: [StandardRequestHandlerTest, TestSortingResponseWriter, TestMinMaxOnMultiValuedField, TestHighFrequencyDictionaryFactory, TestPHPSerializedResponseWriter, PathHierarchyTokenizerFactoryTest, LeaderElectionContextKeyTest, ExplicitHLLTest, AlternateDirectoryTest, UniqFieldsUpdateProcessorFactoryTest, ClusterStateTest, ResourceLoaderTest, DistributedVersionInfoTest, TestQuerySenderNoQuery, SharedFSAutoReplicaFailoverTest, TestFieldSortValues, TestPerFieldSimilarity, SolrRequestParserTest, ExternalFileFieldSortTest, CursorPagingTest, TestDistributedStatsComponentCardinality, DistribDocExpirationUpdateProcessorTest, LeaderElectionIntegrationTest, PolyFieldTest, CursorMarkTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, LeaderInitiatedRecoveryOnCommitTest, HdfsSyncSliceTest, TestHashQParserPlugin, TestSolrXml, TestStressUserVersions, ChangedSchemaMergeTest, CloneFieldUpdateProcessorFactoryTest, ReplicationFactorTest, CurrencyFieldOpenExchangeTest, TestManagedResourceStorage, TestSolrCoreProperties, TestFastWriter, HdfsUnloadDistributedZkTest, TestAuthenticationFramework, RequestLoggingTest, TestLeaderElectionWithEmptyReplica, TestSubQueryTransformerCrossCore, TestStressLucene, PeerSyncTest, CollectionTooManyReplicasTest, TestReplicaProperties, DataDrivenBlockJoinTest, TestLazyCores, TestUseDocValuesAsStored2, BasicZkTest, TestEmbeddedSolrServerConstructors, TestClusterStateMutator, TestSolrCLIRunExample, OutputWriterTest, TestRealTimeGet, TestStressLiveNodes, TestIntervalFaceting, ActionThrottleTest, OverseerStatusTest, TestCollationFieldDocValues, TestDynamicFieldCollectionResource, FastVectorHighlighterTest, DistributedDebugComponentTest, TestScoreJoinQPScore, TestManagedSchemaThreadSafety, XsltUpdateRequestHandlerTest, ReturnFieldsTest, TestRangeQuery, TestJoin, TestCodecSupport, FileBasedSpellCheckerTest, TestFaceting, DistributedSpellCheckComponentTest, RecoveryAfterSoftCommitTest, TestXmlQParserPlugin, TestHighlightDedupGrouping, DistributedFacetPivotLargeTest, HdfsDirectoryFactoryTest, TestSolr4Spatial2, TestSolrDeletionPolicy2, TestNumericTerms32, BlobRepositoryCloudTest, DistanceUnitsTest, TestSearcherReuse, TestLocalFSCloudBackupRestore, TestFastLRUCache, TestSchemaNameResource, EnumFieldTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestMissingGroups, MultiThreadedOCPTest, HLLUtilTest, TestRandomFlRTGCloud, CreateCollectionCleanupTest, TestCrossCoreJoin, TestFieldResource, TestSolrCloudWithDelegationTokens, TestOmitPositions, TestCollectionAPI, TestManagedSynonymFilterFactory, DistributedQueryComponentOptimizationTest, DateFieldTest, ParsingFieldUpdateProcessorsTest, BasicDistributedZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, ShardRoutingTest, TestReplicationHandler]
[junit4] Completed [463/646 (3!)] on J0 in 685.79s, 15 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 60965 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 185 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/185/
8 tests failed.
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:ECF8CD98B5F39A1]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:795)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testUpdateLogSynchronisation(CdcrReplicationDistributedZkTest.java:376)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=SMART}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=ENUM}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=DV}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=STREAM}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=UIF}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib {p0=DVHASH}
Error Message:
mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
Stack Trace:
java.lang.RuntimeException: mismatch: 'A'!='B' @ facets/cat0/buckets/[0]/val
at __randomizedtesting.SeedInfo.seed([F0A0D47A497F1AB0:2C607D0C24FEBE7A]:0)
at org.apache.solr.SolrTestCaseHS.matchJSON(SolrTestCaseHS.java:161)
at org.apache.solr.SolrTestCaseHS.assertJQ(SolrTestCaseHS.java:143)
at org.apache.solr.SolrTestCaseHS$Client$Tester.assertJQ(SolrTestCaseHS.java:255)
at org.apache.solr.SolrTestCaseHS$Client.testJQ(SolrTestCaseHS.java:296)
at org.apache.solr.search.facet.TestJsonFacets.doStatsTemplated(TestJsonFacets.java:1152)
at org.apache.solr.search.facet.TestJsonFacets.doStats(TestJsonFacets.java:361)
at org.apache.solr.search.facet.TestJsonFacets.testStatsDistrib(TestJsonFacets.java:322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:50638
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:50638
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:416)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 47 more
Build Log:
[...truncated 11957 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/init-core-data-001
[junit4] 2> 1810177 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[F0A0D47A497F1AB0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1810178 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[F0A0D47A497F1AB0]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1810180 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1810180 INFO (Thread-15954) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1810180 INFO (Thread-15954) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1810280 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ZkTestServer start zk server on port:55755
[junit4] 2> 1810288 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1810289 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1810290 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1810291 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1810292 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1810292 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1810293 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1810294 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1810295 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1810296 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1810297 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1810306 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 1810307 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 1810308 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
[junit4] 2> 1810309 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1810310 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 1810312 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 1810312 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 1810313 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 1810314 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 1810315 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 1810316 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 1810317 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 1810382 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/cores/collection1
[junit4] 2> 1810384 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1810386 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ccf1013{/,null,AVAILABLE}
[junit4] 2> 1810389 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.ServerConnector Started ServerConnector@370185c0{SSL,[ssl, http/1.1]}{127.0.0.1:35050}
[junit4] 2> 1810389 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server Started @1812857ms
[junit4] 2> 1810389 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/tempDir-001/control/data, hostContext=/, hostPort=35050, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/cores}
[junit4] 2> 1810390 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1810390 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1810390 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1810390 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1810390 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-25T18:38:48.448Z
[junit4] 2> 1810392 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1810392 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/solr.xml
[junit4] 2> 1810396 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1810397 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55755/solr
[junit4] 2> 1810410 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35050_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35050_
[junit4] 2> 1810411 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35050_ ] o.a.s.c.Overseer Overseer (id=96824244845936647-127.0.0.1:35050_-n_0000000000) starting
[junit4] 2> 1810415 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35050_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35050_
[junit4] 2> 1810417 INFO (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1810425 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35050_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/cores
[junit4] 2> 1810425 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35050_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1810427 INFO (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1811437 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1811459 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1811518 WARN (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1811527 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1811534 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1811535 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/control-001/cores/collection1/data/]
[junit4] 2> 1811535 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24047c84
[junit4] 2> 1811535 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=52.87890625, floorSegmentMB=0.60546875, forceMergeDeletesPctAllowed=26.422976436188808, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8169381542443159
[junit4] 2> 1811538 WARN (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1811543 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1811543 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1811543 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1811543 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1811544 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=2.03515625, floorSegmentMB=0.47265625, forceMergeDeletesPctAllowed=22.463816674110646, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1811544 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3a55f04f[collection1] main]
[junit4] 2> 1811544 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1811545 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1811545 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1811545 INFO (searcherExecutor-10000-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3a55f04f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1811545 INFO (coreLoadExecutor-9999-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549187918964195328
[junit4] 2> 1811551 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1811551 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1811551 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:35050/collection1/
[junit4] 2> 1811551 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1811551 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:35050/collection1/ has no replicas
[junit4] 2> 1811553 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:35050/collection1/ shard1
[junit4] 2> 1811704 INFO (coreZkRegister-9992-thread-1-processing-n:127.0.0.1:35050_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:35050_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1811930 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1811931 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1811931 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 1812006 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/cores/collection1
[junit4] 2> 1812006 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001
[junit4] 2> 1812008 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1812009 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cc20e5a{/,null,AVAILABLE}
[junit4] 2> 1812010 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4d5581d5{SSL,[ssl, http/1.1]}{127.0.0.1:50638}
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server Started @1814478ms
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=50638, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/cores}
[junit4] 2> 1812011 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1812011 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-25T18:38:50.069Z
[junit4] 2> 1812013 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1812013 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/solr.xml
[junit4] 2> 1812018 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1812018 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55755/solr
[junit4] 2> 1812024 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1812027 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:50638_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50638_
[junit4] 2> 1812028 INFO (zkCallback-32331-thread-3-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1812028 INFO (zkCallback-32335-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1812029 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1812039 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:50638_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/cores
[junit4] 2> 1812039 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:50638_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1812041 INFO (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1812142 WARN (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.Overseer Bad version writing to ZK using compare-and-set, will force refresh cluster state: KeeperErrorCode = BadVersion for /collections/collection1/state.json
[junit4] 2> 1812144 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1812145 INFO (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1812145 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1813049 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1813059 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1813127 WARN (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1813129 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1813136 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1813136 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 1813136 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24047c84
[junit4] 2> 1813137 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=52.87890625, floorSegmentMB=0.60546875, forceMergeDeletesPctAllowed=26.422976436188808, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8169381542443159
[junit4] 2> 1813140 WARN (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1813146 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1813146 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1813146 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1813146 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1813147 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=2.03515625, floorSegmentMB=0.47265625, forceMergeDeletesPctAllowed=22.463816674110646, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1813147 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ee7d516[collection1] main]
[junit4] 2> 1813148 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1813148 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1813148 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1813149 INFO (searcherExecutor-10011-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ee7d516[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1813149 INFO (coreLoadExecutor-10010-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549187920646111232
[junit4] 2> 1813154 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1813154 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1813154 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50638/collection1/
[junit4] 2> 1813154 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1813154 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:50638/collection1/ has no replicas
[junit4] 2> 1813157 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50638/collection1/ shard2
[junit4] 2> 1813259 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1813308 INFO (coreZkRegister-10005-thread-1-processing-n:127.0.0.1:50638_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:50638_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1813411 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1813615 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/cores/collection1
[junit4] 2> 1813615 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001
[junit4] 2> 1813617 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1813618 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36ac664c{/,null,AVAILABLE}
[junit4] 2> 1813619 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4b39da74{SSL,[ssl, http/1.1]}{127.0.0.1:35810}
[junit4] 2> 1813620 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.e.j.s.Server Started @1816087ms
[junit4] 2> 1813620 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=35810, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/cores}
[junit4] 2> 1813620 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1813620 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1813620 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1813621 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1813621 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-25T18:38:51.679Z
[junit4] 2> 1813622 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1813622 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/solr.xml
[junit4] 2> 1813627 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1813628 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55755/solr
[junit4] 2> 1813633 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35810_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1813636 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35810_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35810_
[junit4] 2> 1813637 INFO (zkCallback-32335-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1813637 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1813638 INFO (zkCallback-32347-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1813638 INFO (zkCallback-32331-thread-1-processing-n:127.0.0.1:35050_) [n:127.0.0.1:35050_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1813654 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35810_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/cores
[junit4] 2> 1813654 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[F0A0D47A497F1AB0]) [n:127.0.0.1:35810_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1813657 INFO (OverseerStateUpdate-96824244845936647-127.0.0.1:35050_-n_0000000000) [n:127.0.0.1:35050_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1813758 INFO (zkCallback-32341-thread-1-processing-n:127.0.0.1:50638_) [n:127.0.0.1:50638_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1813759 INFO (zkCallback-32347-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1814665 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1814675 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1814743 WARN (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1814745 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1814752 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1814752 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_F0A0D47A497F1AB0-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 1814752 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24047c84
[junit4] 2> 1814753 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=9, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=52.87890625, floorSegmentMB=0.60546875, forceMergeDeletesPctAllowed=26.422976436188808, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8169381542443159
[junit4] 2> 1814756 WARN (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1814761 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1814761 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1814762 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1814762 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1814767 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=4, maxMergeAtOnceExplicit=5, maxMergedSegmentMB=2.03515625, floorSegmentMB=0.47265625, forceMergeDeletesPctAllowed=22.463816674110646, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1814767 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@12541ea9[collection1] main]
[junit4] 2> 1814768 INFO (coreLoadExecutor-10021-thread-1-processing-n:127.0.0.1:35810_) [n:127.0.0.1:35810_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.Mana
[...truncated too long message...]
orkspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-6.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_102/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.4.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.suites -stdin
[junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 137. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1481)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] at java.lang.Thread.run(Thread.java:745)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:709: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build.xml:252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:984: At least one slave process threw an exception, first: Forked process returned with error code: 137. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20161025_180835_547.syserr
Total time: 1551 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 184 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/184/
3 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test
Error Message:
There were too many update fails (61 > 60) - we expect it can happen, but shouldn't easily
Stack Trace:
java.lang.AssertionError: There were too many update fails (61 > 60) - we expect it can happen, but shouldn't easily
at __randomizedtesting.SeedInfo.seed([9880255D9B5F0EC5:10D41A8735A3633D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest.test
Error Message:
There were too many update fails (81 > 60) - we expect it can happen, but shouldn't easily
Stack Trace:
java.lang.AssertionError: There were too many update fails (81 > 60) - we expect it can happen, but shouldn't easily
at __randomizedtesting.SeedInfo.seed([9880255D9B5F0EC5:10D41A8735A3633D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:59999
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:59999
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 47 more
Build Log:
[...truncated 11892 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/init-core-data-001
[junit4] 2> 1334310 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1334310 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /p_mt/jm
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1334357 WARN (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1334383 WARN (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1334385 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1334397 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_57354_hdfs____.s9jw2u/webapp
[junit4] 2> 1334714 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57354
[junit4] 2> 1334787 WARN (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1334788 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1334798 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34117_datanode____amchkt/webapp
[junit4] 2> 1335173 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34117
[junit4] 2> 1335242 WARN (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1335243 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1335259 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_49384_datanode____.3bj1ov/webapp
[junit4] 2> 1335324 INFO (IPC Server handler 3 on 56511) [ ] BlockStateChange BLOCK* processReport: from storage DS-69db1c03-b3d7-4447-b0d8-01a22cb8a205 node DatanodeRegistration(127.0.0.1:57613, datanodeUuid=85b1ee6f-b4e3-4386-8145-65154c92ee7d, infoPort=56907, infoSecurePort=0, ipcPort=54108, storageInfo=lv=-56;cid=testClusterID;nsid=938051371;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 1335324 INFO (IPC Server handler 3 on 56511) [ ] BlockStateChange BLOCK* processReport: from storage DS-f5fa1484-9df4-4f04-b96e-60755a97af80 node DatanodeRegistration(127.0.0.1:57613, datanodeUuid=85b1ee6f-b4e3-4386-8145-65154c92ee7d, infoPort=56907, infoSecurePort=0, ipcPort=54108, storageInfo=lv=-56;cid=testClusterID;nsid=938051371;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1335623 INFO (SUITE-HdfsChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:49384
[junit4] 2> 1335715 INFO (IPC Server handler 5 on 56511) [ ] BlockStateChange BLOCK* processReport: from storage DS-02cdec92-441a-495d-a080-8522b8e76a7d node DatanodeRegistration(127.0.0.1:51162, datanodeUuid=450493d1-a8c1-48f6-8fe4-2ba6c4700d54, infoPort=58979, infoSecurePort=0, ipcPort=45505, storageInfo=lv=-56;cid=testClusterID;nsid=938051371;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 1335715 INFO (IPC Server handler 5 on 56511) [ ] BlockStateChange BLOCK* processReport: from storage DS-f7788e2c-3aaa-4ace-80a0-d9e3d1ce52f6 node DatanodeRegistration(127.0.0.1:51162, datanodeUuid=450493d1-a8c1-48f6-8fe4-2ba6c4700d54, infoPort=58979, infoSecurePort=0, ipcPort=45505, storageInfo=lv=-56;cid=testClusterID;nsid=938051371;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1335934 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1335934 INFO (Thread-5775) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1335934 INFO (Thread-5775) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1336034 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkTestServer start zk server on port:56734
[junit4] 2> 1336050 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1336051 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1336053 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1336054 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1336055 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1336056 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1336059 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1336060 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1336062 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1336071 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1336072 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1336144 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/control-001/cores/collection1
[junit4] 2> 1336145 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1336146 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@453bda34{/p_mt/jm,null,AVAILABLE}
[junit4] 2> 1336147 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.ServerConnector Started ServerConnector@5ad4f6ae{HTTP/1.1,[http/1.1]}{127.0.0.1:36622}
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server Started @1338807ms
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:56511/hdfs__localhost_56511__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001_tempDir-002_control_data, hostContext=/p_mt/jm, hostPort=36622, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/control-001/cores}
[junit4] 2> 1336148 ERROR (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1336148 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-24T13:19:53.067Z
[junit4] 2> 1336151 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1336151 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/control-001/solr.xml
[junit4] 2> 1336156 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1336156 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56734/solr
[junit4] 2> 1336171 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36622_p_mt%2Fjm
[junit4] 2> 1336172 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.Overseer Overseer (id=96817328478945284-127.0.0.1:36622_p_mt%2Fjm-n_0000000000) starting
[junit4] 2> 1336176 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36622_p_mt%2Fjm
[junit4] 2> 1336180 INFO (zkCallback-1000-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1336191 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/control-001/cores
[junit4] 2> 1336191 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1336193 INFO (OverseerStateUpdate-96817328478945284-127.0.0.1:36622_p_mt%2Fjm-n_0000000000) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1337206 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1337219 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1337293 WARN (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1337294 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1337306 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1337306 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56511/solr_hdfs_home
[junit4] 2> 1337306 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1337306 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/control-001/cores/collection1], dataDir=[hdfs://localhost:56511/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 1337307 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b8c879c
[junit4] 2> 1337307 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1337319 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1337319 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1337319 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 1338530 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1338532 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1338565 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1338573 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1338573 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1338579 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1338579 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1338608 INFO (IPC Server handler 3 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57613 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f7788e2c-3aaa-4ace-80a0-d9e3d1ce52f6:NORMAL:127.0.0.1:51162|RBW], ReplicaUC[[DISK]DS-f5fa1484-9df4-4f04-b96e-60755a97af80:NORMAL:127.0.0.1:57613|FINALIZED]]} size 0
[junit4] 2> 1338611 INFO (IPC Server handler 7 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51162 is added to blk_1073741825_1001 size 71
[junit4] 2> 1338641 WARN (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1338655 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1338655 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1338655 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1338668 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 1338668 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1338670 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@56b41aa4
[junit4] 2> 1338686 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3d4599cb[collection1] main]
[junit4] 2> 1338687 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1338688 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1338689 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1338691 INFO (searcherExecutor-2739-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3d4599cb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1338691 INFO (coreLoadExecutor-2738-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549077258551951360
[junit4] 2> 1338697 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1338697 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1338698 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36622/p_mt/jm/collection1/
[junit4] 2> 1338698 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1338698 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36622/p_mt/jm/collection1/ has no replicas
[junit4] 2> 1338701 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36622/p_mt/jm/collection1/ shard1
[junit4] 2> 1338852 INFO (coreZkRegister-2731-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:36622_p_mt%2Fjm c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1339203 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1339205 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1339205 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 1339212 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x157f6d8e8170006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1339278 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001/cores/collection1
[junit4] 2> 1339279 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001
[junit4] 2> 1339279 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62f11542{/p_mt/jm,null,AVAILABLE}
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.ServerConnector Started ServerConnector@2902efb1{HTTP/1.1,[http/1.1]}{127.0.0.1:52562}
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server Started @1341943ms
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:56511/hdfs__localhost_56511__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/p_mt/jm, hostPort=52562, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001/cores}
[junit4] 2> 1339284 ERROR (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1339284 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-24T13:19:56.203Z
[junit4] 2> 1339304 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1339304 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001/solr.xml
[junit4] 2> 1339315 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1339319 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56734/solr
[junit4] 2> 1339333 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1339336 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52562_p_mt%2Fjm
[junit4] 2> 1339340 INFO (zkCallback-1004-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1339341 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1339343 INFO (zkCallback-1000-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1339351 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001/cores
[junit4] 2> 1339351 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1339364 INFO (OverseerStateUpdate-96817328478945284-127.0.0.1:36622_p_mt%2Fjm-n_0000000000) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1339470 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1340377 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1340398 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1340446 WARN (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1340447 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1340459 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1340459 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56511/solr_hdfs_home
[junit4] 2> 1340459 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1340459 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:56511/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 1340460 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b8c879c
[junit4] 2> 1340460 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 1340466 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1340466 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1340470 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1340472 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 1340488 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 1340493 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1340493 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1340495 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1340495 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1340527 INFO (IPC Server handler 8 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51162 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f5fa1484-9df4-4f04-b96e-60755a97af80:NORMAL:127.0.0.1:57613|RBW], ReplicaUC[[DISK]DS-f7788e2c-3aaa-4ace-80a0-d9e3d1ce52f6:NORMAL:127.0.0.1:51162|RBW]]} size 0
[junit4] 2> 1340530 INFO (IPC Server handler 9 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57613 is added to blk_1073741826_1002 size 71
[junit4] 2> 1340537 WARN (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1340574 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1340574 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1340574 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1340591 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 1340591 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1340594 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2a041b0e
[junit4] 2> 1340601 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1337e7c6[collection1] main]
[junit4] 2> 1340602 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1340602 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1340602 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1340603 INFO (searcherExecutor-2750-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1337e7c6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1340604 INFO (coreLoadExecutor-2749-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549077260557877248
[junit4] 2> 1340609 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1340609 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1340609 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52562/p_mt/jm/collection1/
[junit4] 2> 1340609 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1340609 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52562/p_mt/jm/collection1/ has no replicas
[junit4] 2> 1340611 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52562/p_mt/jm/collection1/ shard2
[junit4] 2> 1340713 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1340762 INFO (coreZkRegister-2744-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:52562_p_mt%2Fjm c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1340865 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1340923 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001/cores/collection1
[junit4] 2> 1340924 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001
[junit4] 2> 1340924 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1340926 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c792e50{/p_mt/jm,null,AVAILABLE}
[junit4] 2> 1340926 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.ServerConnector Started ServerConnector@7532eec5{HTTP/1.1,[http/1.1]}{127.0.0.1:37326}
[junit4] 2> 1340928 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server Started @1343587ms
[junit4] 2> 1340928 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:56511/hdfs__localhost_56511__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/p_mt/jm, hostPort=37326, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001/cores}
[junit4] 2> 1340928 ERROR (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1340930 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1340930 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1340930 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1340930 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-24T13:19:57.849Z
[junit4] 2> 1340932 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1340932 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001/solr.xml
[junit4] 2> 1340937 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1340937 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56734/solr
[junit4] 2> 1340944 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1340947 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37326_p_mt%2Fjm
[junit4] 2> 1340948 INFO (zkCallback-1000-thread-1-processing-n:127.0.0.1:36622_p_mt%2Fjm) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1340949 INFO (zkCallback-1004-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1340948 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1340959 INFO (zkCallback-1016-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1340963 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001/cores
[junit4] 2> 1340963 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1340964 INFO (OverseerStateUpdate-96817328478945284-127.0.0.1:36622_p_mt%2Fjm-n_0000000000) [n:127.0.0.1:36622_p_mt%2Fjm ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 1341067 INFO (zkCallback-1016-thread-2-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1341067 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1341973 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1341983 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1342041 WARN (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1342042 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1342053 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1342054 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:56511/solr_hdfs_home
[junit4] 2> 1342054 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1342054 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:56511/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 1342054 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3b8c879c
[junit4] 2> 1342054 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 1342061 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1342061 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1342065 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1342066 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 1342081 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:56511/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 1342087 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1342087 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1342090 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1342091 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1342120 INFO (IPC Server handler 6 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57613 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f7788e2c-3aaa-4ace-80a0-d9e3d1ce52f6:NORMAL:127.0.0.1:51162|RBW], ReplicaUC[[DISK]DS-f5fa1484-9df4-4f04-b96e-60755a97af80:NORMAL:127.0.0.1:57613|RBW]]} size 0
[junit4] 2> 1342120 INFO (IPC Server handler 0 on 56511) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51162 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f5fa1484-9df4-4f04-b96e-60755a97af80:NORMAL:127.0.0.1:57613|RBW], ReplicaUC[[DISK]DS-02cdec92-441a-495d-a080-8522b8e76a7d:NORMAL:127.0.0.1:51162|FINALIZED]]} size 0
[junit4] 2> 1342127 WARN (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1342142 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1342142 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1342142 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1342150 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 1342150 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1342152 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@39ca07f5
[junit4] 2> 1342159 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@373ef002[collection1] main]
[junit4] 2> 1342161 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1342161 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1342161 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1342162 INFO (searcherExecutor-2761-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@373ef002[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1342163 INFO (coreLoadExecutor-2760-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1549077262192607232
[junit4] 2> 1342168 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1342168 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1342168 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37326/p_mt/jm/collection1/
[junit4] 2> 1342169 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1342169 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37326/p_mt/jm/collection1/ has no replicas
[junit4] 2> 1342171 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37326/p_mt/jm/collection1/ shard3
[junit4] 2> 1342273 INFO (zkCallback-1016-thread-2-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1342273 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1342322 INFO (coreZkRegister-2755-thread-1-processing-n:127.0.0.1:37326_p_mt%2Fjm x:collection1 s:shard3 c:collection1 r:core_node2) [n:127.0.0.1:37326_p_mt%2Fjm c:collection1 s:shard3 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1342425 INFO (zkCallback-1016-thread-2-processing-n:127.0.0.1:37326_p_mt%2Fjm) [n:127.0.0.1:37326_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1342425 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1342530 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-3-001/cores/collection1
[junit4] 2> 1342531 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-3-001
[junit4] 2> 1342531 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1342532 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@46b8f17b{/p_mt/jm,null,AVAILABLE}
[junit4] 2> 1342533 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.ServerConnector Started ServerConnector@4e68fba4{HTTP/1.1,[http/1.1]}{127.0.0.1:33165}
[junit4] 2> 1342533 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.Server Started @1345192ms
[junit4] 2> 1342533 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:56511/hdfs__localhost_56511__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001_tempDir-002_jetty3, solrconfig=solrconfig.xml, hostContext=/p_mt/jm, hostPort=33165, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-3-001/cores}
[junit4] 2> 1342534 ERROR (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1342535 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 1342535 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1342535 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1342535 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-24T13:19:59.454Z
[junit4] 2> 1342537 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1342537 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001/shard-3-001/solr.xml
[junit4] 2> 1342542 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1342542 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56734/solr
[junit4] 2> 1342549 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:33165_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1342552 INFO (TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [n:127.0.0.1:33165_p_mt%2Fjm ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33165_p_mt%2Fjm
[junit4] 2> 1342554 INFO (zkCallback-1010-thread-1-processing-n:127.0.0.1:52562_p_mt%2Fjm) [n:127.0.0.1:52562_p_mt%2Fjm ] o.a.s.c.c.ZkStateReader Updated live nodes from
[...truncated too long message...]
ner instance=2101852877
[junit4] 2> 2684417 INFO (zkCallback-4046-thread-8-processing-n:127.0.0.1:47736_) [n:127.0.0.1:47736_ ] o.a.s.c.Overseer Overseer (id=96817407308988452-127.0.0.1:47736_-n_0000000010) starting
[junit4] 2> 2684419 INFO (coreCloseExecutor-7966-thread-1) [n:127.0.0.1:47736_ c:collection1 s:shard1 r:core_node10 x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7fb7934d
[junit4] 2> 2684427 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.Overseer Overseer (id=96817407308988452-127.0.0.1:47736_-n_0000000010) closing
[junit4] 2> 2684427 INFO (OverseerStateUpdate-96817407308988452-127.0.0.1:47736_-n_0000000010) [n:127.0.0.1:47736_ ] o.a.s.c.Overseer According to ZK I (id=96817407308988452-127.0.0.1:47736_-n_0000000010) am no longer a leader.
[junit4] 2> 2684427 INFO (OverseerStateUpdate-96817407308988452-127.0.0.1:47736_-n_0000000010) [n:127.0.0.1:47736_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:47736_
[junit4] 2> 2684428 INFO (OverseerCollectionConfigSetProcessor-96817407308988452-127.0.0.1:47736_-n_0000000010) [n:127.0.0.1:47736_ ] o.a.s.c.OverseerTaskProcessor According to ZK I (id=96817407308988452-127.0.0.1:47736_-n_0000000010) am no longer a leader.
[junit4] 2> 2685929 WARN (zkCallback-4046-thread-8-processing-n:127.0.0.1:47736_) [n:127.0.0.1:47736_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2685930 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@5279a6de{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2685930 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6fe91a91{/,null,UNAVAILABLE}
[junit4] 2> 2685932 INFO (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[9880255D9B5F0EC5]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38920 38920
[junit4] 2> 2686100 INFO (Thread-22220) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38920 38920
[junit4] 2> 2686103 WARN (Thread-22220) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 17 /solr/overseer/collection-queue-work
[junit4] 2> 16 /solr/overseer/queue
[junit4] 2> 14 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.method=test -Dtests.seed=9880255D9B5F0EC5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=sv -Dtests.timezone=America/Fortaleza -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 147s J1 | ChaosMonkeyNothingIsSafeTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: There were too many update fails (61 > 60) - we expect it can happen, but shouldn't easily
[junit4] > at __randomizedtesting.SeedInfo.seed([9880255D9B5F0EC5:10D41A8735A3633D]:0)
[junit4] > at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:218)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2686107 INFO (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[9880255D9B5F0EC5]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_9880255D9B5F0EC5-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {rnd_b=PostingsFormat(name=Memory doPackFST= false), _version_=PostingsFormat(name=Direct), a_t=PostingsFormat(name=LuceneFixedGap), a_i=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{}, maxPointsInLeafNode=1467, maxMBSortInHeap=7.299517567334702, sim=ClassicSimilarity, locale=sv, timezone=America/Fortaleza
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=213324432,total=532152320
[junit4] 2> NOTE: All tests run in this JVM: [OverseerRolesTest, SecurityConfHandlerTest, MoreLikeThisHandlerTest, TestSizeLimitedDistributedMap, TestBM25SimilarityFactory, TestStressCloudBlindAtomicUpdates, MultiTermTest, ChangedSchemaMergeTest, ReplaceNodeTest, TestUseDocValuesAsStored2, TestReRankQParserPlugin, IndexBasedSpellCheckerTest, BlockJoinFacetSimpleTest, AtomicUpdatesTest, TestSolr4Spatial2, TestDynamicFieldCollectionResource, SampleTest, TestCoreContainer, DirectSolrSpellCheckerTest, TestStressVersions, TestWordDelimiterFilterFactory, AddBlockUpdateTest, ZkCLITest, TestQueryWrapperFilter, TestCollectionAPI, TestHighFrequencyDictionaryFactory, TestRTGBase, TestManagedSynonymFilterFactory, TestLMDirichletSimilarityFactory, TestSuggestSpellingConverter, TestFastOutputStream, VMParamsZkACLAndCredentialsProvidersTest, TestIntervalFaceting, TestSchemaNameResource, LoggingHandlerTest, TestDownShardTolerantSearch, OverriddenZkACLAndCredentialsProvidersTest, SimpleCollectionCreateDeleteTest, JsonLoaderTest, TestTrackingShardHandlerFactory, SolrInfoMBeanTest, IndexSchemaTest, TestFuzzyAnalyzedSuggestions, TestManagedResourceStorage, JSONWriterTest, DeleteStatusTest, TestStressUserVersions, BJQParserTest, BaseCdcrDistributedZkTest, TestManagedSchemaThreadSafety, TestInitParams, SuggesterFSTTest, DocValuesTest, RequestLoggingTest, TestHdfsBackupRestoreCore, TolerantUpdateProcessorTest, TestTestInjection, DistributedMLTComponentTest, DateMathParserTest, TestFieldResource, ShardSplitTest, SystemInfoHandlerTest, SpellCheckCollatorTest, PreAnalyzedUpdateProcessorTest, DistributedIntervalFacetingTest, TestConfigSetProperties, ShardRoutingCustomTest, OpenCloseCoreStressTest, TestQuerySenderListener, EnumFieldTest, SolrXmlInZkTest, DistributedTermsComponentTest, BasicAuthIntegrationTest, TestNonDefinedSimilarityFactory, AliasIntegrationTest, TestQueryTypes, RulesTest, HdfsChaosMonkeyNothingIsSafeTest, TermsComponentTest, TestFieldCacheVsDocValues, CdcrReplicationHandlerTest, TestUpdate, FileUtilsTest, TestDynamicLoading, DistributedFacetPivotSmallTest, IndexSchemaRuntimeFieldTest, TestElisionMultitermQuery, SolrIndexConfigTest, TestExactSharedStatsCache, TestMissingGroups, TestCursorMarkWithoutUniqueKey, FullHLLTest, DistributedDebugComponentTest, TestSearchPerf, SpellingQueryConverterTest, TestSolrConfigHandlerCloud, TestDistributedStatsComponentCardinality, TestCustomDocTransformer, TestOnReconnectListenerSupport, TestCloudInspectUtil, TermVectorComponentTest, ChaosMonkeySafeLeaderTest, CopyFieldTest, DocExpirationUpdateProcessorFactoryTest, TestHashQParserPlugin, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestJoin, TestReload, HardAutoCommitTest, TestRangeQuery, SolrCoreTest, SpellCheckComponentTest, SolrCmdDistributorTest, QueryElevationComponentTest, PeerSyncTest, BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, TestFunctionQuery, BasicFunctionalityTest, DirectUpdateHandlerTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SolrRequestParserTest, SuggesterTest, TestStressLucene, TestTrie, SpatialFilterTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, QueryEqualityTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestQueryUtils, StandardRequestHandlerTest, TestOmitPositions, DocumentBuilderTest, TestValueSourceCache, PathHierarchyTokenizerFactoryTest, FieldAnalysisRequestHandlerTest, SolrPluginUtilsTest, UpdateRequestProcessorFactoryTest, JSONWriterTest, TestCSVResponseWriter, QueryParsingTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestPartialUpdateDeduplication, CSVRequestHandlerTest, SearchHandlerTest, TestBinaryResponseWriter, TestLFUCache, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, BadComponentTest, TestMergePolicyConfig, TestDocSet, TestBinaryField, MinimalSchemaTest, OutputWriterTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestLuceneMatchVersion, SpellPossibilityIteratorTest, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, ChaosMonkeyNothingIsSafeTest]
[junit4] Completed [440/643 (3!)] on J1 in 147.34s, 1 test, 1 failure <<< FAILURES!
[...truncated 61016 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 183 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/183/
3 tests failed.
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([DEB682AFD519EF2:85BF57F053ADF30A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler
Error Message:
ObjectTracker found 6 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([DEB682AFD519EF2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:261)
at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:34553/_v/la
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:34553/_v/la
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 47 more
Build Log:
[...truncated 11788 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/init-core-data-001
[junit4] 2> 1228375 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 1228375 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1228401 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1228406 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1228408 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1228418 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_43755_hdfs____wdiin6/webapp
[junit4] 2> 1228761 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43755
[junit4] 2> 1228825 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1228826 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1228835 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34935_datanode____.a15lmn/webapp
[junit4] 2> 1229132 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34935
[junit4] 2> 1229192 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1229193 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1229209 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56721_datanode____.4902lg/webapp
[junit4] 2> 1229258 INFO (IPC Server handler 6 on 43100) [ ] BlockStateChange BLOCK* processReport: from storage DS-f3a76652-a6c9-40e2-afbd-26fc1e85078a node DatanodeRegistration(127.0.0.1:57590, datanodeUuid=7bdb3fdf-3c8b-4a06-82d9-9c8764126e8f, infoPort=43119, infoSecurePort=0, ipcPort=52026, storageInfo=lv=-56;cid=testClusterID;nsid=589038129;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1229259 INFO (IPC Server handler 6 on 43100) [ ] BlockStateChange BLOCK* processReport: from storage DS-3db56194-04d2-4c6c-946d-100b010ff998 node DatanodeRegistration(127.0.0.1:57590, datanodeUuid=7bdb3fdf-3c8b-4a06-82d9-9c8764126e8f, infoPort=43119, infoSecurePort=0, ipcPort=52026, storageInfo=lv=-56;cid=testClusterID;nsid=589038129;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1229583 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[DEB682AFD519EF2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56721
[junit4] 2> 1229675 INFO (IPC Server handler 3 on 43100) [ ] BlockStateChange BLOCK* processReport: from storage DS-41898197-9e64-471e-bb64-f871d298929f node DatanodeRegistration(127.0.0.1:43338, datanodeUuid=5ac90a62-2ad1-4eb8-b1b0-9d0c5714694d, infoPort=42655, infoSecurePort=0, ipcPort=37053, storageInfo=lv=-56;cid=testClusterID;nsid=589038129;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1229675 INFO (IPC Server handler 3 on 43100) [ ] BlockStateChange BLOCK* processReport: from storage DS-76e6a292-2b42-42be-bfde-a5ded49278b4 node DatanodeRegistration(127.0.0.1:43338, datanodeUuid=5ac90a62-2ad1-4eb8-b1b0-9d0c5714694d, infoPort=42655, infoSecurePort=0, ipcPort=37053, storageInfo=lv=-56;cid=testClusterID;nsid=589038129;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1229889 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1229889 INFO (Thread-40656) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1229889 INFO (Thread-40656) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1229989 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ZkTestServer start zk server on port:50868
[junit4] 2> 1230001 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1230002 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1230003 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1230004 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1230005 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1230005 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1230006 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1230006 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1230007 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1230008 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1230009 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1230075 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/control-001/cores/collection1
[junit4] 2> 1230076 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29e33dbe{/,null,AVAILABLE}
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.ServerConnector Started ServerConnector@af2f011{HTTP/1.1,[http/1.1]}{127.0.0.1:49162}
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server Started @1232354ms
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=49162, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/control-001/cores}
[junit4] 2> 1230077 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1230077 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-23T13:12:13.699Z
[junit4] 2> 1230085 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1230085 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/control-001/solr.xml
[junit4] 2> 1230089 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1230089 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50868/solr
[junit4] 2> 1230112 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:49162_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:49162_
[junit4] 2> 1230112 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:49162_ ] o.a.s.c.Overseer Overseer (id=96811636065042436-127.0.0.1:49162_-n_0000000000) starting
[junit4] 2> 1230117 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:49162_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49162_
[junit4] 2> 1230118 INFO (zkCallback-1891-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1230136 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:49162_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/control-001/cores
[junit4] 2> 1230136 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:49162_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1230138 INFO (OverseerStateUpdate-96811636065042436-127.0.0.1:49162_-n_0000000000) [n:127.0.0.1:49162_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1231159 WARN (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1231159 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1231171 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1231218 WARN (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1231219 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1231229 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 1231229 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43100/solr_hdfs_home
[junit4] 2> 1231229 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1231230 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/control-001/cores/collection1], dataDir=[hdfs://localhost:43100/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 1231230 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@51d341ab
[junit4] 2> 1231230 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1231235 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1231235 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1231235 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1231241 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1231242 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1231259 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1231846 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1231846 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1231846 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1231851 ERROR (Finalizer) [ ] o.a.s.u.ConcurrentLFUCache ConcurrentLFUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
[junit4] 2> 1231853 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1231853 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@72ab45a4
[junit4] 2> 1231863 INFO (IPC Server handler 6 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43338 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f3a76652-a6c9-40e2-afbd-26fc1e85078a:NORMAL:127.0.0.1:57590|RBW], ReplicaUC[[DISK]DS-41898197-9e64-471e-bb64-f871d298929f:NORMAL:127.0.0.1:43338|RBW]]} size 0
[junit4] 2> 1231864 INFO (IPC Server handler 5 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57590 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-41898197-9e64-471e-bb64-f871d298929f:NORMAL:127.0.0.1:43338|RBW], ReplicaUC[[DISK]DS-3db56194-04d2-4c6c-946d-100b010ff998:NORMAL:127.0.0.1:57590|FINALIZED]]} size 0
[junit4] 2> 1231868 WARN (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1231874 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1231874 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1231874 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1231881 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1231881 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1231881 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4231290808045898]
[junit4] 2> 1231898 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14b52ae3[collection1] main]
[junit4] 2> 1231901 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1231911 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1231911 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1231912 INFO (searcherExecutor-4158-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14b52ae3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1231914 WARN (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37599/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37599 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
[junit4] 2> 1231915 INFO (coreLoadExecutor-4157-thread-1-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548986179164045312
[junit4] 2> 1231923 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1231923 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1231923 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49162/collection1/
[junit4] 2> 1231923 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1231923 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:49162/collection1/ has no replicas
[junit4] 2> 1231926 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49162/collection1/ shard1
[junit4] 2> 1232077 INFO (coreZkRegister-4150-thread-1-processing-n:127.0.0.1:49162_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:49162_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1232349 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1232350 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1232350 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 1232424 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001/cores/collection1
[junit4] 2> 1232425 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001
[junit4] 2> 1232425 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5edc8258{/,null,AVAILABLE}
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.ServerConnector Started ServerConnector@78eb1a1b{HTTP/1.1,[http/1.1]}{127.0.0.1:51733}
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server Started @1234704ms
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=51733, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001/cores}
[junit4] 2> 1232427 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1232427 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1232428 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-23T13:12:16.049Z
[junit4] 2> 1232431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1232431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001/solr.xml
[junit4] 2> 1232436 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1232437 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50868/solr
[junit4] 2> 1232443 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1232445 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:51733_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51733_
[junit4] 2> 1232447 INFO (zkCallback-1895-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1232447 INFO (zkCallback-1891-thread-3-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1232448 INFO (zkCallback-1901-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1232456 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:51733_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001/cores
[junit4] 2> 1232456 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:51733_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1232458 INFO (OverseerStateUpdate-96811636065042436-127.0.0.1:49162_-n_0000000000) [n:127.0.0.1:49162_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1232560 WARN (OverseerStateUpdate-96811636065042436-127.0.0.1:49162_-n_0000000000) [n:127.0.0.1:49162_ ] o.a.s.c.Overseer Bad version writing to ZK using compare-and-set, will force refresh cluster state: KeeperErrorCode = BadVersion for /collections/collection1/state.json
[junit4] 2> 1232562 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1232562 INFO (OverseerStateUpdate-96811636065042436-127.0.0.1:49162_-n_0000000000) [n:127.0.0.1:49162_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1232563 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1233465 WARN (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1233466 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1233475 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1233532 WARN (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1233533 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1233543 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1233543 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43100/solr_hdfs_home
[junit4] 2> 1233543 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1233544 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:43100/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 1233544 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@51d341ab
[junit4] 2> 1233544 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 1233550 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1233550 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1233550 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1233553 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1233554 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 1233565 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 1233570 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1233570 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1233570 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1233573 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1233573 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6629decd
[junit4] 2> 1233581 INFO (IPC Server handler 7 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57590 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-41898197-9e64-471e-bb64-f871d298929f:NORMAL:127.0.0.1:43338|RBW], ReplicaUC[[DISK]DS-f3a76652-a6c9-40e2-afbd-26fc1e85078a:NORMAL:127.0.0.1:57590|FINALIZED]]} size 0
[junit4] 2> 1233582 INFO (IPC Server handler 4 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43338 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f3a76652-a6c9-40e2-afbd-26fc1e85078a:NORMAL:127.0.0.1:57590|FINALIZED], ReplicaUC[[DISK]DS-76e6a292-2b42-42be-bfde-a5ded49278b4:NORMAL:127.0.0.1:43338|FINALIZED]]} size 0
[junit4] 2> 1233588 WARN (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1233601 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1233601 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1233601 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1233607 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1233607 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1233609 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4231290808045898]
[junit4] 2> 1233613 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6f7e0a6b[collection1] main]
[junit4] 2> 1233614 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1233614 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1233614 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1233615 WARN (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37599/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37599 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
[junit4] 2> 1233617 INFO (searcherExecutor-4169-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6f7e0a6b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1233617 WARN (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37599/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37599 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
[junit4] 2> 1233617 INFO (coreLoadExecutor-4168-thread-1-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548986180948721664
[junit4] 2> 1233623 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1233623 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1233623 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51733/collection1/
[junit4] 2> 1233623 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1233623 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51733/collection1/ has no replicas
[junit4] 2> 1233626 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51733/collection1/ shard2
[junit4] 2> 1233729 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1233779 INFO (coreZkRegister-4163-thread-1-processing-n:127.0.0.1:51733_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:51733_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1233881 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1234020 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001/cores/collection1
[junit4] 2> 1234021 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001
[junit4] 2> 1234021 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 1234022 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@18e68912{/,null,AVAILABLE}
[junit4] 2> 1234022 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.ServerConnector Started ServerConnector@3b6e4dea{HTTP/1.1,[http/1.1]}{127.0.0.1:41698}
[junit4] 2> 1234022 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.Server Started @1236300ms
[junit4] 2> 1234023 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/, hostPort=41698, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001/cores}
[junit4] 2> 1234023 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1234023 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.3.0
[junit4] 2> 1234023 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1234023 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1234023 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-23T13:12:17.645Z
[junit4] 2> 1234026 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1234026 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001/solr.xml
[junit4] 2> 1234030 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1234031 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50868/solr
[junit4] 2> 1234040 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:41698_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1234043 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:41698_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41698_
[junit4] 2> 1234044 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1234044 INFO (zkCallback-1895-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1234044 INFO (zkCallback-1891-thread-3-processing-n:127.0.0.1:49162_) [n:127.0.0.1:49162_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1234044 INFO (zkCallback-1907-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1234065 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:41698_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001/cores
[junit4] 2> 1234065 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[DEB682AFD519EF2]) [n:127.0.0.1:41698_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1234066 INFO (OverseerStateUpdate-96811636065042436-127.0.0.1:49162_-n_0000000000) [n:127.0.0.1:49162_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1234168 INFO (zkCallback-1901-thread-2-processing-n:127.0.0.1:51733_) [n:127.0.0.1:51733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1234168 INFO (zkCallback-1907-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1235075 WARN (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1235076 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 1235088 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1235157 WARN (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 1235159 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43100/solr_hdfs_home
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_DEB682AFD519EF2-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:43100/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@51d341ab
[junit4] 2> 1235170 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 1235180 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1235180 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1235180 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1235184 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1235187 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 1235203 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43100/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 1235209 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1235209 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1235209 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1235212 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1235212 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@67370cca
[junit4] 2> 1235229 INFO (IPC Server handler 3 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43338 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3db56194-04d2-4c6c-946d-100b010ff998:NORMAL:127.0.0.1:57590|RBW], ReplicaUC[[DISK]DS-41898197-9e64-471e-bb64-f871d298929f:NORMAL:127.0.0.1:43338|FINALIZED]]} size 0
[junit4] 2> 1235230 INFO (IPC Server handler 9 on 43100) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57590 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3db56194-04d2-4c6c-946d-100b010ff998:NORMAL:127.0.0.1:57590|RBW], ReplicaUC[[DISK]DS-41898197-9e64-471e-bb64-f871d298929f:NORMAL:127.0.0.1:43338|FINALIZED]]} size 0
[junit4] 2> 1235234 WARN (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 1235249 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1235249 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1235249 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1235256 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1235256 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1235258 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4231290808045898]
[junit4] 2> 1235265 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5ddc2a47[collection1] main]
[junit4] 2> 1235265 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1235266 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1235266 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1235267 WARN (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37599/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37599 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
[junit4] 2> 1235267 WARN (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.h.HdfsLocalityReporter Could not retrieve locality information for hdfs://localhost:37599/solr3 due to exception: java.net.ConnectException: Call From lucene1-us-west/10.41.0.5 to localhost:37599 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
[junit4] 2> 1235267 INFO (coreLoadExecutor-4179-thread-1-processing-n:127.0.0.1:41698_) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548986182678872064
[junit4] 2> 1235268 INFO (searcherExecutor-4180-thread-1-processing-n:127.0.0.1:41698_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:41698_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5ddc2a47[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1235275 INFO (coreZkRegister-4174-thread-1-processing-n:127.0.0.1:41698_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:
[...truncated too long message...]
ntextHandler Stopped o.e.j.s.ServletContextHandler@36ccd71f{/solr,null,UNAVAILABLE}
[junit4] 2> 2653535 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.ServerConnector Stopped ServerConnector@ffe0809{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2653535 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[DEB682AFD519EF2]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=890108088
[junit4] 2> 2653535 INFO (coreCloseExecutor-5957-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@45c7c30c
[junit4] 2> 2653536 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigReplication-seed#[DEB682AFD519EF2]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7433064{/solr,null,UNAVAILABLE}
[junit4] 2> 2653540 INFO (SUITE-TestReplicationHandler-seed#[DEB682AFD519EF2]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {name=FSTOrd50, id=FSTOrd50, newname=FSTOrd50}, docValues:{}, maxPointsInLeafNode=1519, maxMBSortInHeap=5.248649075273284, sim=ClassicSimilarity, locale=fr-CA, timezone=Antarctica/Macquarie
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=1,free=208423720,total=464519168
[junit4] 2> NOTE: All tests run in this JVM: [TestCursorMarkWithoutUniqueKey, SystemInfoHandlerTest, TestFieldSortValues, SolrIndexConfigTest, OverseerRolesTest, TestSimpleTrackingShardHandler, TestReRankQParserPlugin, TestCharFilters, TestFieldCacheVsDocValues, DistributedQueryComponentOptimizationTest, IndexSchemaRuntimeFieldTest, SortSpecParsingTest, TestLuceneMatchVersion, ConvertedLegacyTest, TestXmlQParserPlugin, JSONWriterTest, DocumentBuilderTest, JsonLoaderTest, TestShardHandlerFactory, TestPKIAuthenticationPlugin, TestSortingResponseWriter, ShardRoutingCustomTest, SoftAutoCommitTest, TestBlendedInfixSuggestions, TestMissingGroups, TestRequestStatusCollectionAPI, TestLeaderRecoverFromLogOnStartup, TestHashPartitioner, HdfsTlogReplayBufferedWhileIndexingTest, TestDynamicLoading, TestSlowCompositeReaderWrapper, DeleteStatusTest, TestBackupRepositoryFactory, TestDistributedGrouping, SolrInfoMBeanTest, TestTestInjection, TestSystemIdResolver, ZkStateWriterTest, TestQueryUtils, TestMinMaxOnMultiValuedField, TestQueryWrapperFilter, VMParamsZkACLAndCredentialsProvidersTest, HdfsRestartWhileUpdatingTest, BaseCdcrDistributedZkTest, DataDrivenBlockJoinTest, SolrCmdDistributorTest, ShardRoutingTest, EchoParamsTest, TestSolrConfigHandlerCloud, SpellCheckComponentTest, CustomCollectionTest, TermsComponentTest, DocExpirationUpdateProcessorFactoryTest, AtomicUpdatesTest, AliasIntegrationTest, SimpleCollectionCreateDeleteTest, TestHighFrequencyDictionaryFactory, TestMiniSolrCloudCluster, TestQuerySenderListener, ClusterStateUpdateTest, TestHdfsUpdateLog, TestHdfsBackupRestoreCore, TestUseDocValuesAsStored2, BasicFunctionalityTest, DocValuesTest, SimpleMLTQParserTest, DistributedIntervalFacetingTest, TestIntervalFaceting, TestScoreJoinQPScore, TestCustomDocTransformer, RulesTest, TestTolerantUpdateProcessorRandomCloud, AddBlockUpdateTest, SearchHandlerTest, TestOnReconnectListenerSupport, MigrateRouteKeyTest, TestManagedResourceStorage, DistribJoinFromCollectionTest, FullHLLTest, LoggingHandlerTest, CollectionsAPIAsyncDistributedZkTest, SolrIndexSplitterTest, TestSuggestSpellingConverter, DistributedDebugComponentTest, TestHashQParserPlugin, CSVRequestHandlerTest, ZkCLITest, SampleTest, PreAnalyzedUpdateProcessorTest, TestCollectionAPI, SortByFunctionTest, TestCoreContainer, TestCollationFieldDocValues, TestCSVResponseWriter, SimplePostToolTest, BlockJoinFacetSimpleTest, DirectSolrConnectionTest, TestFieldCacheSanityChecker, URLClassifyProcessorTest, ZkControllerTest, RequestLoggingTest, CollectionReloadTest, TestSolrQueryParserResource, LeaderFailureAfterFreshStartTest, TestStressVersions, SparseHLLTest, UnloadDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, TestReplicationHandler]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestReplicationHandler -Dtests.seed=DEB682AFD519EF2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=fr-CA -Dtests.timezone=Antarctica/Macquarie -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestReplicationHandler (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ObjectTracker found 6 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:369)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([DEB682AFD519EF2]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:261)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] Completed [456/643 (3!)] on J1 in 603.01s, 15 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 60962 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 182 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/182/
5 tests failed.
FAILED: org.apache.lucene.search.TestFuzzyQuery.testRandom
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([C9BA1A19CF61CE25]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([C9BA1A19CF61CE25]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart
Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1
Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
at __randomizedtesting.SeedInfo.seed([D2DA772529CDA0CA:8EC790A4E7ED14FC]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:795)
at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Could not find collection:delete_data_dir
Stack Trace:
java.lang.AssertionError: Could not find collection:delete_data_dir
at __randomizedtesting.SeedInfo.seed([D2DA772529CDA0CA:5A8E48FF8731CD32]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:153)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:138)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=15038, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15038, name=collection2, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41861/_/a
at __randomizedtesting.SeedInfo.seed([D2DA772529CDA0CA]:0)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41861/_/a
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 11 more
Build Log:
[...truncated 1785 lines...]
[junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> oct 22, 2016 1:36:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestFuzzyQuery
[junit4] 2> 1) Thread[id=5526, name=TEST-TestFuzzyQuery.testRandom-seed#[C9BA1A19CF61CE25], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.System.identityHashCode(Native Method)
[junit4] 2> at java.util.IdentityHashMap.hash(IdentityHashMap.java:294)
[junit4] 2> at java.util.IdentityHashMap.get(IdentityHashMap.java:328)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:239)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=5525, name=SUITE-TestFuzzyQuery-seed#[C9BA1A19CF61CE25], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:690)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:687)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:687)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:703)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:683)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:243)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:354)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:10)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.method=testRandom -Dtests.seed=C9BA1A19CF61CE25 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=es-PR -Dtests.timezone=Europe/Brussels -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7200s J2 | TestFuzzyQuery.testRandom <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([C9BA1A19CF61CE25]:0)
[junit4] 2> oct 22, 2016 1:36:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> oct 22, 2016 1:36:24 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestFuzzyQuery:
[junit4] 2> 1) Thread[id=5526, name=TEST-TestFuzzyQuery.testRandom-seed#[C9BA1A19CF61CE25], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.util.WeakHashMap.getTable(WeakHashMap.java:350)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:427)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> oct 22, 2016 1:36:24 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=5526, name=TEST-TestFuzzyQuery.testRandom-seed#[C9BA1A19CF61CE25], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> oct 22, 2016 1:36:27 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=5526, name=TEST-TestFuzzyQuery.testRandom-seed#[C9BA1A19CF61CE25], state=RUNNABLE, group=TGRP-TestFuzzyQuery]
[junit4] 2> at java.lang.Object.hashCode(Native Method)
[junit4] 2> at java.util.WeakHashMap.hash(WeakHashMap.java:298)
[junit4] 2> at java.util.WeakHashMap.getEntry(WeakHashMap.java:426)
[junit4] 2> at java.util.WeakHashMap.containsKey(WeakHashMap.java:417)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.context(RandomizedContext.java:258)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedContext.current(RandomizedContext.java:134)
[junit4] 2> at org.apache.lucene.util.LuceneTestCase.random(LuceneTestCase.java:729)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.randomSimpleString(TestFuzzyQuery.java:506)
[junit4] 2> at org.apache.lucene.search.TestFuzzyQuery.testRandom(TestFuzzyQuery.java:517)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {field=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, maxPointsInLeafNode=498, maxMBSortInHeap=7.645477083263929, sim=ClassicSimilarity, locale=es-PR, timezone=Europe/Brussels
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_102 (64-bit)/cpus=4,threads=2,free=253467920,total=353370112
[junit4] 2> NOTE: All tests run in this JVM: [TestCharTermAttributeImpl, TestSparseFixedBitDocIdSet, TestSpanExplanations, TestLongPostings, TestNeedsScores, TestSpanNearQuery, TestSetOnce, TestOperations, TestDisjunctionMaxQuery, TestLucene50StoredFieldsFormatHighCompression, TestBM25Similarity, TestAllFilesCheckIndexHeader, TestSort, FiniteStringsIteratorTest, TestMergePolicyWrapper, TestOfflineSorter, TestNamedSPILoader, TestSpanTermQuery, TestIndexWriterDeleteByQuery, TestTimeLimitingCollector, TestSameScoresWithThreads, TestMergeSchedulerExternal, TestForceMergeForever, TestReaderWrapperDVTypeCheck, Test2BPostingsBytes, TestVirtualMethod, TestDocInverterPerFieldErrorInfo, TestCharsRef, TestIndexWriterMaxDocs, TestExceedMaxTermLength, TestMultiCollector, TestLucene62SegmentInfoFormat, TestSynonymQuery, TestNRTThreads, TestIndexWriterThreadsToSegments, Test2BPoints, TestTermVectorsWriter, TestIndexCommit, TestSimpleFSLockFactory, TestIntroSelector, TestNativeFSLockFactory, TestGeoUtils, Test2BPostings, TestPositionIncrement, TestSpanSearchEquivalence, TestRecyclingIntBlockAllocator, TestNRTCachingDirectory, TestByteBlockPool, TestRollingUpdates, TestMultiLevelSkipList, TestTopDocsCollector, TestOrdinalMap, TestReqExclBulkScorer, TestCachingTokenFilter, TestLazyProxSkipping, TestSortedNumericSortField, TestSpanExplanationsOfNonMatches, TestTieredMergePolicy, TestNoMergePolicy, TestWildcard, TestStressNRT, TestMultiValuedNumericRangeQuery, TestPayloadsOnVectors, TestIndexWriterMerging, TestToken, TestIOUtils, TestNorms, TestDocumentWriter, TestSentinelIntSet, TestLRUQueryCache, TestBKD, TestIndexWriterReader, TestDeletionPolicy, TestBooleanOr, TestBasics, TestSloppyPhraseQuery, TestIndexWriterMergePolicy, TestPersistentSnapshotDeletionPolicy, TestAtomicUpdate, TestStressAdvance, TestStandardAnalyzer, TestIndexWriterCommit, TestSearchWithThreads, TestSearchAfter, TestConcurrentMergeScheduler, TestHighCompressionMode, TestDocumentsWriterStallControl, TestCustomSearcherSort, TestTermsEnum2, TestSpans, TestPostingsOffsets, TestMixedCodecs, TestDeterminism, TestDocsAndPositions, TestBagOfPostings, TestIndexInput, TestOmitTf, TestMultiFields, TestMultiTermConstantScore, TestFuzzyQuery]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFuzzyQuery -Dtests.seed=C9BA1A19CF61CE25 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/test-data/enwiki.random.lines.txt -Dtests.locale=es-PR -Dtests.timezone=Europe/Brussels -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | TestFuzzyQuery (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([C9BA1A19CF61CE25]:0)
[junit4] Completed [439/439 (1!)] on J2 in 7223.19s, 9 tests, 2 errors <<< FAILURES!
[...truncated 9468 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/init-core-data-001
[junit4] 2> 77303 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 77303 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 77938 WARN (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 78610 WARN (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 78773 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 78791 WARN (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 78847 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 78887 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_35621_hdfs____.ck4q9v/webapp
[junit4] 2> 79784 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35621
[junit4] 2> 81037 WARN (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 81045 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 81061 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35651_datanode____.jjxfu1/webapp
[junit4] 2> 81517 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35651
[junit4] 2> 82211 WARN (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 82212 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 82237 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58292_datanode____3wyxwv/webapp
[junit4] 2> 83102 INFO (SUITE-StressHdfsTest-seed#[D2DA772529CDA0CA]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58292
[junit4] 2> 84270 INFO (IPC Server handler 6 on 48602) [ ] BlockStateChange BLOCK* processReport: from storage DS-deb7c69a-f4e0-445d-b900-1989d3b06c5d node DatanodeRegistration(127.0.0.1:51251, datanodeUuid=044e2ad9-77cd-4f9b-b617-2c44ddd36195, infoPort=50966, infoSecurePort=0, ipcPort=47888, storageInfo=lv=-56;cid=testClusterID;nsid=850042227;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 84270 INFO (IPC Server handler 6 on 48602) [ ] BlockStateChange BLOCK* processReport: from storage DS-da34d1a6-5483-4412-b482-e8e07fcb2a74 node DatanodeRegistration(127.0.0.1:51251, datanodeUuid=044e2ad9-77cd-4f9b-b617-2c44ddd36195, infoPort=50966, infoSecurePort=0, ipcPort=47888, storageInfo=lv=-56;cid=testClusterID;nsid=850042227;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
[junit4] 2> 84271 INFO (IPC Server handler 7 on 48602) [ ] BlockStateChange BLOCK* processReport: from storage DS-b03408a3-f225-42a4-869a-5da64299fc1b node DatanodeRegistration(127.0.0.1:57053, datanodeUuid=2127a5bf-7c8a-4e42-9039-46928478e735, infoPort=57491, infoSecurePort=0, ipcPort=49698, storageInfo=lv=-56;cid=testClusterID;nsid=850042227;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 84271 INFO (IPC Server handler 7 on 48602) [ ] BlockStateChange BLOCK* processReport: from storage DS-49ab18b3-3912-4392-93eb-b8c1a65e5059 node DatanodeRegistration(127.0.0.1:57053, datanodeUuid=2127a5bf-7c8a-4e42-9039-46928478e735, infoPort=57491, infoSecurePort=0, ipcPort=49698, storageInfo=lv=-56;cid=testClusterID;nsid=850042227;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 84572 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 84587 INFO (Thread-214) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 84587 INFO (Thread-214) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 84687 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ZkTestServer start zk server on port:38882
[junit4] 2> 84731 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 84733 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 84736 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 84737 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 84739 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 84741 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 84744 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 84745 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 84746 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 84747 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 84748 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 84836 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/control-001/cores/collection1
[junit4] 2> 84838 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 84848 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@129c3964{/,null,AVAILABLE}
[junit4] 2> 84855 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.ServerConnector Started ServerConnector@17b48b{HTTP/1.1,[http/1.1]}{127.0.0.1:37879}
[junit4] 2> 84855 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server Started @87385ms
[junit4] 2> 84855 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:48602/hdfs__localhost_48602__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001_tempDir-002_control_data, hostContext=/, hostPort=37879, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/control-001/cores}
[junit4] 2> 84855 ERROR (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 84855 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 84856 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 84856 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 84856 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-22T13:30:53.799Z
[junit4] 2> 84865 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 84865 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/control-001/solr.xml
[junit4] 2> 84880 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 84880 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38882/solr
[junit4] 2> 84912 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:37879_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37879_
[junit4] 2> 84913 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:37879_ ] o.a.s.c.Overseer Overseer (id=96806047156273156-127.0.0.1:37879_-n_0000000000) starting
[junit4] 2> 84921 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:37879_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37879_
[junit4] 2> 84923 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 84934 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:37879_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/control-001/cores
[junit4] 2> 84934 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:37879_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 84963 INFO (OverseerStateUpdate-96806047156273156-127.0.0.1:37879_-n_0000000000) [n:127.0.0.1:37879_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 85988 WARN (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 85988 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 86003 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 86087 WARN (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 86092 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 86104 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 86109 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:48602/solr_hdfs_home
[junit4] 2> 86109 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 86110 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/control-001/cores/collection1], dataDir=[hdfs://localhost:48602/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 86110 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@b015906
[junit4] 2> 86110 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 86128 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 86128 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 86129 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 87338 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 87343 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 87393 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 87409 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 87409 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 87409 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 87423 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 87423 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=21.2705078125, floorSegmentMB=1.9169921875, forceMergeDeletesPctAllowed=3.8468100692602833, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 87633 INFO (IPC Server handler 6 on 48602) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57053 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-b03408a3-f225-42a4-869a-5da64299fc1b:NORMAL:127.0.0.1:57053|RBW], ReplicaUC[[DISK]DS-da34d1a6-5483-4412-b482-e8e07fcb2a74:NORMAL:127.0.0.1:51251|RBW]]} size 71
[junit4] 2> 87634 INFO (IPC Server handler 9 on 48602) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51251 is added to blk_1073741825_1001 size 71
[junit4] 2> 88063 WARN (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 88076 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 88076 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 88076 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 88088 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 88088 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 88090 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=73.0458984375, floorSegmentMB=1.1533203125, forceMergeDeletesPctAllowed=22.549457164613965, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2531707549830936
[junit4] 2> 88165 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@684a457f[collection1] main]
[junit4] 2> 88166 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 88166 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 88168 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 88170 INFO (searcherExecutor-207-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@684a457f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 88170 INFO (coreLoadExecutor-206-thread-1-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548896758255321088
[junit4] 2> 88177 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 88177 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 88178 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37879/collection1/
[junit4] 2> 88178 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 88178 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37879/collection1/ has no replicas
[junit4] 2> 88182 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37879/collection1/ shard1
[junit4] 2> 88333 INFO (coreZkRegister-199-thread-1-processing-n:127.0.0.1:37879_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:37879_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 88472 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 88473 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 88473 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 88546 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001/cores/collection1
[junit4] 2> 88547 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001
[junit4] 2> 88548 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 88549 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f303c18{/,null,AVAILABLE}
[junit4] 2> 88550 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.ServerConnector Started ServerConnector@3941de25{HTTP/1.1,[http/1.1]}{127.0.0.1:52002}
[junit4] 2> 88550 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server Started @91079ms
[junit4] 2> 88550 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:48602/hdfs__localhost_48602__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=52002, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001/cores}
[junit4] 2> 88550 ERROR (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 88551 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 88551 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 88551 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 88551 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-22T13:30:57.494Z
[junit4] 2> 88554 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 88554 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001/solr.xml
[junit4] 2> 88560 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 88560 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38882/solr
[junit4] 2> 88568 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 88571 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:52002_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52002_
[junit4] 2> 88572 INFO (zkCallback-108-thread-3-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 88573 INFO (zkCallback-112-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 88574 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 88588 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:52002_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001/cores
[junit4] 2> 88588 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:52002_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 88590 INFO (OverseerStateUpdate-96806047156273156-127.0.0.1:37879_-n_0000000000) [n:127.0.0.1:37879_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 88697 WARN (OverseerStateUpdate-96806047156273156-127.0.0.1:37879_-n_0000000000) [n:127.0.0.1:37879_ ] o.a.s.c.Overseer Bad version writing to ZK using compare-and-set, will force refresh cluster state: KeeperErrorCode = BadVersion for /collections/collection1/state.json
[junit4] 2> 88701 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 88702 INFO (OverseerStateUpdate-96806047156273156-127.0.0.1:37879_-n_0000000000) [n:127.0.0.1:37879_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 88703 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 89617 WARN (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 89618 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 89630 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 89712 WARN (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 89714 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 89724 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 89725 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:48602/solr_hdfs_home
[junit4] 2> 89725 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 89725 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:48602/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 89725 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@b015906
[junit4] 2> 89725 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 89734 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 89734 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 89734 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 89742 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 89743 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 89783 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:48602/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 89791 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 89791 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 89791 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 89800 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 89800 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=21.2705078125, floorSegmentMB=1.9169921875, forceMergeDeletesPctAllowed=3.8468100692602833, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 89846 INFO (IPC Server handler 0 on 48602) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57053 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-da34d1a6-5483-4412-b482-e8e07fcb2a74:NORMAL:127.0.0.1:51251|RBW], ReplicaUC[[DISK]DS-49ab18b3-3912-4392-93eb-b8c1a65e5059:NORMAL:127.0.0.1:57053|FINALIZED]]} size 0
[junit4] 2> 89850 INFO (IPC Server handler 1 on 48602) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:51251 is added to blk_1073741826_1002 size 71
[junit4] 2> 89875 WARN (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 89942 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 89942 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 89942 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 89955 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 89955 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 89961 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=73.0458984375, floorSegmentMB=1.1533203125, forceMergeDeletesPctAllowed=22.549457164613965, segmentsPerTier=49.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2531707549830936
[junit4] 2> 89987 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@237d98b8[collection1] main]
[junit4] 2> 89991 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 89991 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 89992 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 89994 INFO (searcherExecutor-218-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@237d98b8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 89994 INFO (coreLoadExecutor-217-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548896760167923712
[junit4] 2> 90007 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 90007 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 90007 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52002/collection1/
[junit4] 2> 90007 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 90007 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52002/collection1/ has no replicas
[junit4] 2> 90011 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52002/collection1/ shard1
[junit4] 2> 90112 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 90162 INFO (coreZkRegister-212-thread-1-processing-n:127.0.0.1:52002_ x:collection1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:52002_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 90170 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-2-001/cores/collection1
[junit4] 2> 90170 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-2-001
[junit4] 2> 90171 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 90172 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@119d3f5a{/,null,AVAILABLE}
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.ServerConnector Started ServerConnector@36391eeb{HTTP/1.1,[http/1.1]}{127.0.0.1:60060}
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.e.j.s.Server Started @92703ms
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:48602/hdfs__localhost_48602__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=60060, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-2-001/cores}
[junit4] 2> 90173 ERROR (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 90173 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-22T13:30:59.116Z
[junit4] 2> 90184 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 90184 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-2-001/solr.xml
[junit4] 2> 90189 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 90190 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38882/solr
[junit4] 2> 90198 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:60060_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 90206 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:60060_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60060_
[junit4] 2> 90207 INFO (zkCallback-112-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 90211 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 90211 INFO (zkCallback-108-thread-2-processing-n:127.0.0.1:37879_) [n:127.0.0.1:37879_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 90212 INFO (zkCallback-124-thread-1-processing-n:127.0.0.1:60060_) [n:127.0.0.1:60060_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 90215 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:52002_) [n:127.0.0.1:52002_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 90221 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:60060_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D2DA772529CDA0CA-001/shard-2-001/cores
[junit4] 2> 90221 INFO (TEST-StressHdfsTest.test-seed#[D2DA772529CDA0CA]) [n:127.0.0.1:60060_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 90224 INFO (OverseerStateUpdate-96806047156273156-127.0.0.1:37879_-n_0000000000) [n:127.0.0.1:37879_ ] o.a.s.c.o.ReplicaMutator Assigning new node
[...truncated too long message...]
cheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-6.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_102/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.4.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20161022_132926_361.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20161022_132926_361.suites -stdin
[junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] at java.lang.Thread.run(Thread.java:745)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:709: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build.xml:252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
Total time: 351 minutes 59 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 181 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/181/
1 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=26637, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=26637, name=collection4, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:41100
at __randomizedtesting.SeedInfo.seed([DCAD7898587F04D9]:0)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:997)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:41100
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:604)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:262)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1292)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1062)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1004)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:498)
... 11 more
Build Log:
[...truncated 12218 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/init-core-data-001
[junit4] 2> 2665484 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[DCAD7898587F04D9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2665485 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[DCAD7898587F04D9]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2665487 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2665487 INFO (Thread-12164) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2665487 INFO (Thread-12164) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2665587 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkTestServer start zk server on port:34046
[junit4] 2> 2665598 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2665600 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2665602 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2665604 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2665605 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2665607 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2665608 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2665610 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2665611 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2665612 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2665614 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2665623 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 2665624 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 2665626 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf2/enumsConfig.xml
[junit4] 2> 2665627 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2665629 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 2665630 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 2665631 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 2665632 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 2665634 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 2665635 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 2665636 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 2665641 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 2665758 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/cores/collection1
[junit4] 2> 2665760 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2665761 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ad2062a{/,null,AVAILABLE}
[junit4] 2> 2665763 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.ServerConnector Started ServerConnector@54796a97{SSL,[ssl, http/1.1]}{127.0.0.1:44523}
[junit4] 2> 2665764 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server Started @2668987ms
[junit4] 2> 2665764 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/tempDir-001/control/data, hostContext=/, hostPort=44523, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/cores}
[junit4] 2> 2665764 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2665765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 2665765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2665765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2665765 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-21T00:59:33.838Z
[junit4] 2> 2665774 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2665775 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/solr.xml
[junit4] 2> 2665781 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2665782 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34046/solr
[junit4] 2> 2665803 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:44523_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44523_
[junit4] 2> 2665804 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:44523_ ] o.a.s.c.Overseer Overseer (id=96797430484893703-127.0.0.1:44523_-n_0000000000) starting
[junit4] 2> 2665813 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:44523_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44523_
[junit4] 2> 2665816 INFO (zkCallback-3046-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2665831 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:44523_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/cores
[junit4] 2> 2665831 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:44523_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2665834 INFO (OverseerStateUpdate-96797430484893703-127.0.0.1:44523_-n_0000000000) [n:127.0.0.1:44523_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2665852 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 2665866 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2665962 WARN (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2665964 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2665976 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2665976 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/control-001/cores/collection1/data/]
[junit4] 2> 2665976 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e59acf
[junit4] 2> 2665977 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39906220184106955]
[junit4] 2> 2665981 WARN (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2665989 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2665989 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2665990 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2665990 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2665990 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@42aaac9
[junit4] 2> 2665990 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@65eba463[collection1] main]
[junit4] 2> 2665991 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2665992 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2665992 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2665992 INFO (searcherExecutor-7228-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@65eba463[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2665993 INFO (coreLoadExecutor-7227-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548758888287830016
[junit4] 2> 2666001 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2666001 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2666001 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:44523/collection1/
[junit4] 2> 2666001 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2666001 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:44523/collection1/ has no replicas
[junit4] 2> 2666005 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:44523/collection1/ shard1
[junit4] 2> 2666156 INFO (coreZkRegister-7220-thread-1-processing-n:127.0.0.1:44523_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:44523_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2666336 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2666338 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2666439 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/cores/collection1
[junit4] 2> 2666439 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001
[junit4] 2> 2666441 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2666442 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ff5bf8a{/,null,AVAILABLE}
[junit4] 2> 2666444 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.ServerConnector Started ServerConnector@2b02e071{SSL,[ssl, http/1.1]}{127.0.0.1:41100}
[junit4] 2> 2666444 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server Started @2669668ms
[junit4] 2> 2666444 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=41100, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/cores}
[junit4] 2> 2666445 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2666445 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 2666445 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2666445 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2666445 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-21T00:59:34.518Z
[junit4] 2> 2666448 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2666448 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/solr.xml
[junit4] 2> 2666454 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2666455 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34046/solr
[junit4] 2> 2666464 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2666469 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41100_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41100_
[junit4] 2> 2666470 INFO (zkCallback-3046-thread-3-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2666474 INFO (zkCallback-3050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2666475 INFO (zkCallback-3055-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2666490 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41100_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/cores
[junit4] 2> 2666490 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41100_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2666492 INFO (OverseerStateUpdate-96797430484893703-127.0.0.1:44523_-n_0000000000) [n:127.0.0.1:44523_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 2667507 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 2667521 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2667611 WARN (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2667614 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2667625 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2667626 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 2667626 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e59acf
[junit4] 2> 2667627 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39906220184106955]
[junit4] 2> 2667631 WARN (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2667640 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2667640 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2667640 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2667640 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2667642 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1166eaee
[junit4] 2> 2667642 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5bef58c6[collection1] main]
[junit4] 2> 2667643 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2667644 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2667644 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2667645 INFO (searcherExecutor-7239-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5bef58c6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2667645 INFO (coreLoadExecutor-7238-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548758890020077568
[junit4] 2> 2667652 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2667652 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2667652 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:41100/collection1/
[junit4] 2> 2667652 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2667652 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:41100/collection1/ has no replicas
[junit4] 2> 2667656 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:41100/collection1/ shard2
[junit4] 2> 2667808 INFO (coreZkRegister-7233-thread-1-processing-n:127.0.0.1:41100_ x:collection1 s:shard2 c:collection1 r:core_node1) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2668111 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/cores/collection1
[junit4] 2> 2668112 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001
[junit4] 2> 2668113 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2668115 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@39c9ac60{/,null,AVAILABLE}
[junit4] 2> 2668117 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.ServerConnector Started ServerConnector@782856da{SSL,[ssl, http/1.1]}{127.0.0.1:38915}
[junit4] 2> 2668118 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server Started @2671342ms
[junit4] 2> 2668118 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=38915, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/cores}
[junit4] 2> 2668118 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2668119 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 2668119 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2668119 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2668119 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-21T00:59:36.192Z
[junit4] 2> 2668122 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2668122 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/solr.xml
[junit4] 2> 2668129 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2668130 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34046/solr
[junit4] 2> 2668141 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:38915_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2668146 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:38915_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38915_
[junit4] 2> 2668148 INFO (zkCallback-3055-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2668149 INFO (zkCallback-3050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2668149 INFO (zkCallback-3046-thread-2-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2668150 INFO (zkCallback-3061-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2668160 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:38915_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/cores
[junit4] 2> 2668160 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:38915_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2668162 INFO (OverseerStateUpdate-96797430484893703-127.0.0.1:44523_-n_0000000000) [n:127.0.0.1:44523_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2669176 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 2669191 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2669295 WARN (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2669298 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2669311 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2669312 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 2669312 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e59acf
[junit4] 2> 2669313 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39906220184106955]
[junit4] 2> 2669318 WARN (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2669328 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2669328 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2669329 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2669329 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2669331 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@16ea032
[junit4] 2> 2669331 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@25ad21d1[collection1] main]
[junit4] 2> 2669333 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2669333 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2669333 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2669334 INFO (searcherExecutor-7250-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25ad21d1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2669335 INFO (coreLoadExecutor-7249-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548758891792171008
[junit4] 2> 2669343 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2669343 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2669343 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38915/collection1/
[junit4] 2> 2669343 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2669343 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:38915/collection1/ has no replicas
[junit4] 2> 2669348 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38915/collection1/ shard1
[junit4] 2> 2669499 INFO (coreZkRegister-7244-thread-1-processing-n:127.0.0.1:38915_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:38915_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2669785 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/cores/collection1
[junit4] 2> 2669786 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001
[junit4] 2> 2669788 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2669790 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@84a9b27{/,null,AVAILABLE}
[junit4] 2> 2669791 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.ServerConnector Started ServerConnector@57c27552{SSL,[ssl, http/1.1]}{127.0.0.1:41240}
[junit4] 2> 2669792 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server Started @2673016ms
[junit4] 2> 2669792 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=41240, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/cores}
[junit4] 2> 2669792 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2669793 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 2669793 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2669793 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2669793 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-21T00:59:37.866Z
[junit4] 2> 2669797 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2669797 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/solr.xml
[junit4] 2> 2669804 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2669805 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34046/solr
[junit4] 2> 2669816 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41240_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2669822 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41240_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41240_
[junit4] 2> 2669824 INFO (zkCallback-3055-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2669824 INFO (zkCallback-3046-thread-1-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2669825 INFO (zkCallback-3061-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2669825 INFO (zkCallback-3050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2669826 INFO (zkCallback-3067-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2669852 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41240_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/cores
[junit4] 2> 2669852 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:41240_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2669854 INFO (OverseerStateUpdate-96797430484893703-127.0.0.1:44523_-n_0000000000) [n:127.0.0.1:44523_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 2670873 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 2670893 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2670995 WARN (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2670998 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2671010 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2671011 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 2671011 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e59acf
[junit4] 2> 2671012 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39906220184106955]
[junit4] 2> 2671017 WARN (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2671027 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2671027 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2671028 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2671028 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2671029 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7f87d759
[junit4] 2> 2671030 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3bb167c6[collection1] main]
[junit4] 2> 2671031 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2671032 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2671032 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2671033 INFO (searcherExecutor-7261-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3bb167c6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2671033 INFO (coreLoadExecutor-7260-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548758893572653056
[junit4] 2> 2671038 INFO (coreZkRegister-7255-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2671038 INFO (updateExecutor-3064-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2671039 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2671039 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2671039 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2671039 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2671039 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:41100/collection1/] and I am [https://127.0.0.1:41240/collection1/]
[junit4] 2> 2671041 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:41100]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:41240_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2671102 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2671103 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 2671103 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:41240_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"https://127.0.0.1:41240","node_name":"127.0.0.1:41240_","state":"down"}
[junit4] 2> 2671476 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/cores/collection1
[junit4] 2> 2671477 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001
[junit4] 2> 2671478 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 2671481 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@10400459{/,null,AVAILABLE}
[junit4] 2> 2671483 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.ServerConnector Started ServerConnector@144cf26{SSL,[ssl, http/1.1]}{127.0.0.1:37384}
[junit4] 2> 2671484 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.e.j.s.Server Started @2674708ms
[junit4] 2> 2671484 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=37384, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/cores}
[junit4] 2> 2671484 ERROR (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2671485 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.3.0
[junit4] 2> 2671485 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2671485 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2671485 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2016-10-21T00:59:39.558Z
[junit4] 2> 2671488 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2671489 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/solr.xml
[junit4] 2> 2671496 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2671496 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34046/solr
[junit4] 2> 2671509 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:37384_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2671518 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:37384_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37384_
[junit4] 2> 2671519 INFO (zkCallback-3055-thread-1-processing-n:127.0.0.1:41100_) [n:127.0.0.1:41100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671519 INFO (zkCallback-3061-thread-1-processing-n:127.0.0.1:38915_) [n:127.0.0.1:38915_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671522 INFO (zkCallback-3050-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671522 INFO (zkCallback-3046-thread-3-processing-n:127.0.0.1:44523_) [n:127.0.0.1:44523_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671519 INFO (zkCallback-3067-thread-1-processing-n:127.0.0.1:41240_) [n:127.0.0.1:41240_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671525 INFO (zkCallback-3074-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2671538 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:37384_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/cores
[junit4] 2> 2671538 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [n:127.0.0.1:37384_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2671539 INFO (OverseerStateUpdate-96797430484893703-127.0.0.1:44523_-n_0000000000) [n:127.0.0.1:44523_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2672103 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:41240_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"https://127.0.0.1:41240","node_name":"127.0.0.1:41240_","state":"recovering"}
[junit4] 2> 2672103 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2672103 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:41240_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 2672553 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.3.0
[junit4] 2> 2672579 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2672672 WARN (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2672675 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2672688 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2672688 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_DCAD7898587F04D9-001/shard-4-001/cores/collection1/data/]
[junit4] 2> 2672688 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2e59acf
[junit4] 2> 2672689 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.39906220184106955]
[junit4] 2> 2672695 WARN (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
[junit4] 2> 2672704 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2672705 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2672705 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2672706 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2672706 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@48d464a6
[junit4] 2> 2672706 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@553676ec[collection1] main]
[junit4] 2> 2672707 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2672708 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2672708 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2672709 INFO (searcherExecutor-7272-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@553676ec[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2672709 INFO (coreLoadExecutor-7271-thread-1-processing-n:127.0.0.1:37384_) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1548758895330066432
[junit4] 2> 2672714 INFO (coreZkRegister-7266-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2672715 INFO (updateExecutor-3071-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2672715 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2672716 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2672716 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2672716 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2672720 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:38915/collection1/] and I am [https://127.0.0.1:37384/collection1/]
[junit4] 2> 2672722 INFO (recoveryExecutor-3072-thread-1-processing-n:127.0.0.1:37384_ x:collection1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:37384_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:38915]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:37384_&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2672789 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2672790 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 2672790 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:37384_, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1","base_url":"https://127.0.0.1:37384","node_name":"127.0.0.1:37384_","state":"down"}
[junit4] 2> 2673041 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2673042 INFO (TEST-CollectionsAPIDistributedZkTest.test-seed#[DCAD7898587F04D9]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4] 2> 2673790 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:37384_, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{"core":"collection1","base_url":"https://127.0.0.1:37384","node_name":"127.0.0.1:37384_","state":"recovering"}
[junit4] 2> 2673791 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2673791 INFO (qtp2147037837-26175) [n:127.0.0.1:38915_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:37384_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
[junit4] 2> 2679104 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:41100/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2679105 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:41240_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=https://127.0.0.1:41240 START replicas=[https://127.0.0.1:41100/collection1/] nUpdates=100
[junit4] 2> 2679113 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2679113 INFO (qtp51286503-26147) [n:127.0.0.1:41100_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2679114 INFO (recoveryExecutor-3065-thread-1-processing-n:127.0.0.1:41240_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:
[...truncated too long message...]
acheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-6.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_102/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20161021_001504_788.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20161021_001504_788.suites -stdin
[junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] at java.lang.Thread.run(Thread.java:745)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:773: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:709: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/build.xml:252: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/checkout/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
Total time: 269 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any