You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/06/14 09:13:17 UTC
[JENKINS] Lucene-Solr-Tests-6.x - Build # 271 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/271/
All tests passed
Build Log:
[...truncated 63246 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj1561999643
[ecj-lint] Compiling 932 source files to /tmp/ecj1561999643
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/Assign.java (at line 101)
[ecj-lint] Collections.sort(shardIdNames, (o1, o2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] 2. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/Assign.java (at line 101)
[ecj-lint] Collections.sort(shardIdNames, (o1, o2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] 3. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/Assign.java (at line 101)
[ecj-lint] Collections.sort(shardIdNames, (o1, o2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 4. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/rule/ReplicaAssigner.java (at line 213)
[ecj-lint] Collections.sort(sortedLiveNodes, (n1, n2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] 5. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/rule/ReplicaAssigner.java (at line 213)
[ecj-lint] Collections.sort(sortedLiveNodes, (n1, n2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] 6. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/cloud/rule/ReplicaAssigner.java (at line 213)
[ecj-lint] Collections.sort(sortedLiveNodes, (n1, n2) -> {
[ecj-lint] ^^^^^^^^^^^
[ecj-lint] (Recovered) Internal inconsistency detected during lambda shape analysis
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 7. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/core/HdfsDirectoryFactory.java (at line 34)
[ecj-lint] import org.apache.hadoop.fs.FsStatus;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.hadoop.fs.FsStatus is never used
[ecj-lint] ----------
[ecj-lint] 8. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/core/HdfsDirectoryFactory.java (at line 227)
[ecj-lint] dir = new BlockDirectory(path, hdfsDir, cache, null, blockCacheReadEnabled, false, cacheMerges, cacheReadOnce);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'dir' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 9. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 120)
[ecj-lint] reader = cfiltfac.create(reader);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'reader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 10. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/AnalysisRequestHandlerBase.java (at line 144)
[ecj-lint] return namedList;
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'listBasedTokenStream' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 11. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 79)
[ecj-lint] import org.apache.solr.core.DirectoryFactory;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.core.DirectoryFactory is never used
[ecj-lint] ----------
[ecj-lint] 12. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java (at line 1187)
[ecj-lint] DirectoryReader reader = s==null ? null : s.get().getIndexReader();
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'reader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 13. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/SQLHandler.java (at line 254)
[ecj-lint] ParallelStream parallelStream = new ParallelStream(workerZkHost, workerCollection, tupleStream, numWorkers, comp);
[ecj-lint] ^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'parallelStream' is never closed
[ecj-lint] ----------
[ecj-lint] 14. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/SQLHandler.java (at line 393)
[ecj-lint] ParallelStream parallelStream = new ParallelStream(workerZkHost, workerCollection, tupleStream, numWorkers, comp);
[ecj-lint] ^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'parallelStream' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 15. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/handler/component/ExpandComponent.java (at line 223)
[ecj-lint] UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getLeafReader(), field), mapping);
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'uninvertingReader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 16. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/highlight/DefaultSolrHighlighter.java (at line 544)
[ecj-lint] tvWindowStream = new OffsetWindowTokenFilter(tvStream);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'tvWindowStream' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 17. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/request/SimpleFacets.java (at line 829)
[ecj-lint] fastForRandomSet = new HashDocSet(sset.getDocs(), 0, sset.size());
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'fastForRandomSet' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 18. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/schema/OpenExchangeRatesOrgProvider.java (at line 146)
[ecj-lint] ratesJsonStream = resourceLoader.openResource(ratesFileLocation);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'ratesJsonStream' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 19. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/CollapsingQParserPlugin.java (at line 1187)
[ecj-lint] UninvertingReader uninvertingReader = new UninvertingReader(new ReaderWrapper(searcher.getLeafReader(), collapseField), mapping);
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'uninvertingReader' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 20. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 145)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 21. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 154)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] 22. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/DocSlice.java (at line 163)
[ecj-lint] HashDocSet h = new HashDocSet(docs,offset,len);
[ecj-lint] ^
[ecj-lint] Resource leak: 'h' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 23. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/function/OrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/search/function/ReverseOrdFieldSource.java (at line 78)
[ecj-lint] SolrIndexSearcher is = (SolrIndexSearcher) o;
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 25. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java (at line 184)
[ecj-lint] throw new SolrException(ErrorCode.SERVER_ERROR, "Error occurred while loading solr.xml from zookeeper", e);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'loader' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/update/CdcrTransactionLog.java (at line 98)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/update/TransactionLog.java (at line 226)
[ecj-lint] ChannelFastInputStream is = new ChannelFastInputStream(channel, pos);
[ecj-lint] ^^
[ecj-lint] Resource leak: 'is' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/util/SimplePostTool.java (at line 640)
[ecj-lint] BAOS bos = new BAOS();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'bos' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/core/src/java/org/apache/solr/util/SolrLogLayout.java (at line 149)
[ecj-lint] SolrQueryRequest req = requestInfo == null ? null : requestInfo.getReq();
[ecj-lint] ^^^
[ecj-lint] Resource leak: 'req' is never closed
[ecj-lint] ----------
[ecj-lint] 31 problems (2 errors, 29 warnings)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:740: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build.xml:652: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:1976: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:2015: Compile failed; see the compiler error output for details.
Total time: 83 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-6.x - Build # 272 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/272/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud: 1) Thread[id=2728, name=OverseerHdfsCoreFailoverThread-96077765416058894-127.0.0.1:45530_solr-n_0000000002, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud:
1) Thread[id=2728, name=OverseerHdfsCoreFailoverThread-96077765416058894-127.0.0.1:45530_solr-n_0000000002, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2728, name=OverseerHdfsCoreFailoverThread-96077765416058894-127.0.0.1:45530_solr-n_0000000002, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.fillInStackTrace(Throwable.java:783) at java.lang.Throwable.<init>(Throwable.java:265) at java.lang.Exception.<init>(Exception.java:66) at java.lang.InterruptedException.<init>(InterruptedException.java:67) at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2728, name=OverseerHdfsCoreFailoverThread-96077765416058894-127.0.0.1:45530_solr-n_0000000002, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
at java.lang.Throwable.<init>(Throwable.java:265)
at java.lang.Exception.<init>(Exception.java:66)
at java.lang.InterruptedException.<init>(InterruptedException.java:67)
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestBulkSchemaConcurrent
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.schema.TestBulkSchemaConcurrent: 1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495) at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299) at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestBulkSchemaConcurrent:
1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299)
at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.schema.TestBulkSchemaConcurrent
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent] at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:170) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543) at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409) at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177) at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304) at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495) at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299) at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent]
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299)
at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)
Build Log:
[...truncated 10781 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/init-core-data-001
[junit4] 2> 411312 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-9182 - causes OOM)
[junit4] 2> 411312 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.TestTolerantUpdateProcessorRandomCloud Configuring cluster: servers=7, shards=2, repfactor=3
[junit4] 2> 411312 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 411312 INFO (Thread-1105) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 411312 INFO (Thread-1105) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 411412 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41333
[junit4] 2> 411413 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411414 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411419 INFO (zkCallback-385-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3f00ac4c name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411419 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411420 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411420 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[9CBE136414B5BF3A]-worker) [ ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
[junit4] 2> 411438 INFO (jetty-launcher-384-thread-1) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411439 INFO (jetty-launcher-384-thread-2) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411446 INFO (jetty-launcher-384-thread-4) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411460 INFO (jetty-launcher-384-thread-3) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411464 INFO (jetty-launcher-384-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f006f51{/solr,null,AVAILABLE}
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.e.j.s.ServerConnector Started ServerConnector@468a48a2{HTTP/1.1,[http/1.1]}{127.0.0.1:54972}
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.e.j.s.Server Started @415939ms
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54972}
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node4'
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411466 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411479 INFO (jetty-launcher-384-thread-5) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411480 INFO (jetty-launcher-384-thread-6) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411483 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411485 INFO (jetty-launcher-384-thread-7) [ ] o.e.j.s.Server jetty-9.3.8.v20160314
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f37c065{/solr,null,AVAILABLE}
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.e.j.s.ServerConnector Started ServerConnector@615a897a{HTTP/1.1,[http/1.1]}{127.0.0.1:54590}
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.e.j.s.Server Started @416005ms
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54590}
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6'
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411531 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411532 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411537 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28997655{/solr,null,AVAILABLE}
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.e.j.s.ServerConnector Started ServerConnector@17c253cd{HTTP/1.1,[http/1.1]}{127.0.0.1:46752}
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.e.j.s.Server Started @416014ms
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46752}
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node5'
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411540 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411541 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411550 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411553 INFO (jetty-launcher-384-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@400aee34{/solr,null,AVAILABLE}
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.e.j.s.ServerConnector Started ServerConnector@6e2982e5{HTTP/1.1,[http/1.1]}{127.0.0.1:45530}
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.e.j.s.Server Started @416027ms
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45530}
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2'
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411554 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411557 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411570 INFO (zkCallback-386-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6473c93c name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411570 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411570 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411570 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411575 INFO (zkCallback-387-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f5b8aaf name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411575 INFO (jetty-launcher-384-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cc315ab{/solr,null,AVAILABLE}
[junit4] 2> 411575 INFO (jetty-launcher-384-thread-3) [ ] o.e.j.s.ServerConnector Started ServerConnector@17597a75{HTTP/1.1,[http/1.1]}{127.0.0.1:35845}
[junit4] 2> 411575 INFO (jetty-launcher-384-thread-3) [ ] o.e.j.s.Server Started @416049ms
[junit4] 2> 411575 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35845}
[junit4] 2> 411575 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411576 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3'
[junit4] 2> 411576 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411576 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411576 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411578 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411632 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411632 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411632 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411638 INFO (jetty-launcher-384-thread-7) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e50609d{/solr,null,AVAILABLE}
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.e.j.s.ServerConnector Started ServerConnector@13f122c8{HTTP/1.1,[http/1.1]}{127.0.0.1:36578}
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.e.j.s.Server Started @416112ms
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36578}
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node7'
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411639 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411640 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4368fc4c{/solr,null,AVAILABLE}
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.e.j.s.ServerConnector Started ServerConnector@28be91d2{HTTP/1.1,[http/1.1]}{127.0.0.1:40200}
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.e.j.s.Server Started @416114ms
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40200}
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node1'
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 411641 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411643 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411644 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6/.
[junit4] 2> 411644 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.CoreContainer New CoreContainer 1597826312
[junit4] 2> 411644 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6]
[junit4] 2> 411644 WARN (jetty-launcher-384-thread-6) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6/lib
[junit4] 2> 411658 INFO (zkCallback-389-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d495886 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411658 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411658 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411658 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411658 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411659 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411669 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node4/.
[junit4] 2> 411669 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.CoreContainer New CoreContainer 1240043773
[junit4] 2> 411669 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node4]
[junit4] 2> 411669 WARN (jetty-launcher-384-thread-4) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node4/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node4/lib
[junit4] 2> 411669 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411669 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411668 INFO (zkCallback-390-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6c04d39c name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411670 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411670 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411670 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411661 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411673 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411673 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411673 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411673 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411673 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411668 INFO (zkCallback-391-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1bcfd6ae name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411683 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411684 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411683 INFO (zkCallback-398-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@63f61ff2 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411683 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2/.
[junit4] 2> 411684 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.CoreContainer New CoreContainer 530086824
[junit4] 2> 411684 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2]
[junit4] 2> 411684 WARN (jetty-launcher-384-thread-2) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2/lib
[junit4] 2> 411684 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411685 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411683 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3/.
[junit4] 2> 411689 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.CoreContainer New CoreContainer 1223856207
[junit4] 2> 411689 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3]
[junit4] 2> 411689 WARN (jetty-launcher-384-thread-3) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3/lib
[junit4] 2> 411690 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411690 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411691 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411691 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411691 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411691 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411691 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411678 INFO (zkCallback-395-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53385e65 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411677 INFO (zkCallback-388-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@390370e7 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411700 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411700 INFO (jetty-launcher-384-thread-6) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411677 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411700 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411700 INFO (zkCallback-404-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@795ec65e name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411700 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411674 INFO (zkCallback-392-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@43ae2bca name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411694 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-4) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411701 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411702 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@5af91b73
[junit4] 2> 411703 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f6700008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411689 INFO (zkCallback-401-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2ccc2ae4 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411688 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411684 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 411711 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@550086e2
[junit4] 2> 411703 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411713 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411713 INFO (jetty-launcher-384-thread-2) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411714 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411714 INFO (jetty-launcher-384-thread-3) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411715 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f6700003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411716 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f670000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411726 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@59812181
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@3bd5bd93
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node1/.
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.CoreContainer New CoreContainer 133532461
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node1]
[junit4] 2> 411727 WARN (jetty-launcher-384-thread-1) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node1/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node1/lib
[junit4] 2> 411735 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411727 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node7/.
[junit4] 2> 411735 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.CoreContainer New CoreContainer 922893507
[junit4] 2> 411735 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node7]
[junit4] 2> 411735 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411735 WARN (jetty-launcher-384-thread-7) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node7/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node7/lib
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411743 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411743 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411743 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411743 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411743 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411743 INFO (zkCallback-405-thread-1-processing-n:127.0.0.1:54972_solr) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@623c380c name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411738 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node5/.
[junit4] 2> 411784 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411784 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.CoreContainer New CoreContainer 1427280490
[junit4] 2> 411784 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node5]
[junit4] 2> 411784 WARN (jetty-launcher-384-thread-5) [ ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node5/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node5/lib
[junit4] 2> 411784 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
[junit4] 2> 411738 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411784 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41333/solr
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.ZkController zkHost includes chroot
[junit4] 2> 411785 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
[junit4] 2> 411736 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411782 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411787 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411787 INFO (zkCallback-408-thread-1-processing-n:127.0.0.1:45530_solr) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d0ef0d5 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411787 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411787 INFO (zkCallback-406-thread-1-processing-n:127.0.0.1:54590_solr) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@18ff4104 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411787 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411788 INFO (zkCallback-407-thread-1-processing-n:127.0.0.1:35845_solr) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@25ff7838 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411788 INFO (zkCallback-414-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2503bb9 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411788 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411788 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411788 INFO (zkCallback-411-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@67f5d662 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411788 INFO (jetty-launcher-384-thread-7) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411788 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411788 INFO (jetty-launcher-384-thread-1) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411789 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 411789 INFO (zkCallback-417-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@8f99ea4 name:ZooKeeperConnection Watcher:127.0.0.1:41333 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411789 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411789 INFO (jetty-launcher-384-thread-5) [ ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
[junit4] 2> 411789 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f6700010, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411790 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@2caa258b
[junit4] 2> 411790 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f6700011, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411790 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 411790 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@353a7520
[junit4] 2> 411791 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 411791 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
[junit4] 2> 411791 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1555637f6700012, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 411793 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 411793 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 411793 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
[junit4] 2> 411795 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 411795 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 411795 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 411795 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
[junit4] 2> 411796 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 411796 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 411796 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 411796 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
[junit4] 2> 411797 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 411797 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 411797 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
[junit4] 2> 411798 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 411798 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 411798 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 411798 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /collections
[junit4] 2> 411799 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 411799 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 411799 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 411799 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
[junit4] 2> 411800 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 411800 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 411800 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 411800 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
[junit4] 2> 411801 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 411801 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 411801 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 411801 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /security.json
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411802 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411803 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 411803 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 411803 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 411803 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
[junit4] 2> 411805 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 411805 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 411805 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 411805 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
[junit4] 2> 411806 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 411806 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 411806 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 411806 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
[junit4] 2> 411807 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411807 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411807 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411810 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058892-127.0.0.1:54972_solr-n_0000000000
[junit4] 2> 411810 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058894-127.0.0.1:45530_solr-n_0000000002
[junit4] 2> 411810 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058893-127.0.0.1:54590_solr-n_0000000001
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058895-127.0.0.1:35845_solr-n_0000000003
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54972_solr
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058893-127.0.0.1:54590_solr-n_0000000001 to know if I could be the leader
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058892-127.0.0.1:54972_solr-n_0000000000 to know if I could be the leader
[junit4] 2> 411811 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54590_solr
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058894-127.0.0.1:45530_solr-n_0000000002 to know if I could be the leader
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45530_solr
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:54590_solr
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35845_solr
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45530_solr
[junit4] 2> 411812 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:35845_solr
[junit4] 2> 411813 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer Overseer (id=96077765416058892-127.0.0.1:54972_solr-n_0000000000) starting
[junit4] 2> 411814 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
[junit4] 2> 411819 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411819 INFO (zkCallback-408-thread-1-processing-n:127.0.0.1:45530_solr) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
[junit4] 2> 411819 INFO (zkCallback-406-thread-1-processing-n:127.0.0.1:54590_solr) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
[junit4] 2> 411819 INFO (zkCallback-405-thread-1-processing-n:127.0.0.1:54972_solr) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
[junit4] 2> 411819 INFO (zkCallback-407-thread-1-processing-n:127.0.0.1:35845_solr) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
[junit4] 2> 411819 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$89/1834104950@5652106e
[junit4] 2> 411821 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411821 INFO (zkCallback-418-thread-1-processing-n:127.0.0.1:40200_solr) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@de964f name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411822 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 411822 INFO (zkCallback-406-thread-1-processing-n:127.0.0.1:54590_solr) [n:127.0.0.1:54590_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411822 INFO (zkCallback-407-thread-1-processing-n:127.0.0.1:35845_solr) [n:127.0.0.1:35845_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411823 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411828 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 411828 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 411828 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node3/.
[junit4] 2> 411828 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
[junit4] 2> 411854 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:46752_solr) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7547776a name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411856 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411856 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 411860 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2
[junit4] 2> 411860 INFO (jetty-launcher-384-thread-3) [n:127.0.0.1:35845_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 411861 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411861 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411862 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411863 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411863 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058899-127.0.0.1:40200_solr-n_0000000004
[junit4] 2> 411863 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058895-127.0.0.1:35845_solr-n_0000000003 to know if I could be the leader
[junit4] 2> 411863 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:40200_solr as DOWN
[junit4] 2> 411874 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411874 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411874 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411876 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411879 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 411879 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 411879 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node6/.
[junit4] 2> 411880 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
[junit4] 2> 411880 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058901-127.0.0.1:46752_solr-n_0000000005
[junit4] 2> 411882 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058899-127.0.0.1:40200_solr-n_0000000004 to know if I could be the leader
[junit4] 2> 411882 INFO (jetty-launcher-384-thread-5) [n:127.0.0.1:46752_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:46752_solr as DOWN
[junit4] 2> 411883 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
[junit4] 2> 411883 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.CoreContainer No authentication plugin used.
[junit4] 2> 411883 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_9CBE136414B5BF3A-001/tempDir-001/node2/.
[junit4] 2> 411883 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
[junit4] 2> 411902 INFO (OverseerCollectionConfigSetProcessor-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 411903 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 411914 INFO (jetty-launcher-384-thread-4) [n:127.0.0.1:54972_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:54972_solr as DOWN
[junit4] 2> 411918 INFO (zkCallback-405-thread-1-processing-n:127.0.0.1:54972_solr) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411918 INFO (zkCallback-408-thread-1-processing-n:127.0.0.1:45530_solr) [n:127.0.0.1:45530_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411918 INFO (zkCallback-419-thread-1-processing-n:127.0.0.1:36578_solr) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7eebddc9 name:ZooKeeperConnection Watcher:127.0.0.1:41333/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 411918 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 411921 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper...
[junit4] 2> 411921 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
[junit4] 2> 411921 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 411922 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411922 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96077765416058900-127.0.0.1:36578_solr-n_0000000006
[junit4] 2> 411923 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96077765416058901-127.0.0.1:46752_solr-n_0000000005 to know if I could be the leader
[junit4] 2> 411923 INFO (jetty-launcher-384-thread-7) [n:127.0.0.1:36578_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:36578_solr as DOWN
[junit4] 2> 411945 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (3)
[junit4] 2> 411945 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:40200_solr"} current state version: 0
[junit4] 2> 411945 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40200_solr
[junit4] 2> 411946 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:46752_solr"} current state version: 0
[junit4] 2> 411946 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46752_solr
[junit4] 2> 411946 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:54972_solr"} current state version: 0
[junit4] 2> 411946 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:54972_solr
[junit4] 2> 411947 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:36578_solr"} current state version: 0
[junit4] 2> 411947 INFO (OverseerStateUpdate-96077765416058892-127.0.0.1:54972_solr-n_0000000000) [n:127.0.0.1:54972_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:36578_solr
[junit4] 2> 412000 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2
[junit4] 2> 412001 INFO (jetty-launcher-384-thread-2) [n:127.0.0.1:45530_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 412001 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build/solr-core/test/J2
[junit4] 2> 412001 INFO (jetty-launcher-384-thread-6) [n:127.0.0.1:54590_solr ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
[junit4] 2> 412864 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40200_solr
[junit4] 2> 412864 INFO (jetty-launcher-384-thread-1) [n:127.0.0.1:40200_solr ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0
[...truncated too long message...]
a$8/1371611390.run(Unknown Source)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=872, maxMBSortInHeap=6.1489913269382335, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=es-SV, timezone=America/Shiprock
[junit4] 2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=193926496,total=390594560
[junit4] 2> NOTE: All tests run in this JVM: [JavabinLoaderTest, TestMissingGroups, TestSolr4Spatial, SolrRequestParserTest, IndexSchemaTest, TestManagedStopFilterFactory, RequiredFieldsTest, TestLeaderElectionZkExpiry, CacheHeaderTest, TestOmitPositions, TestCollapseQParserPlugin, TestAnalyzedSuggestions, DistributedQueryElevationComponentTest, DeleteLastCustomShardedReplicaTest, TestBulkSchemaConcurrent]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBulkSchemaConcurrent -Dtests.seed=9CBE136414B5BF3A -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-SV -Dtests.timezone=America/Shiprock -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J0 | TestBulkSchemaConcurrent (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.schema.TestBulkSchemaConcurrent:
[junit4] > 1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent]
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:170)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
[junit4] > at sun.security.ssl.InputRecord.read(InputRecord.java:503)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
[junit4] > at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
[junit4] > at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=702, name=httpUriRequest-129-thread-1-processing-x:collection1 r:core_node3 n:127.0.0.1:35591_quo%2Fap https:////127.0.0.1:49901//quo//ap//collection1 s:shard2 c:collection1, state=RUNNABLE, group=TGRP-TestBulkSchemaConcurrent]
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:170)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
[junit4] > at sun.security.ssl.InputRecord.read(InputRecord.java:503)
[junit4] > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
[junit4] > at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
[junit4] > at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
[junit4] > at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
[junit4] > at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
[junit4] > at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
[junit4] > at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
[junit4] > at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
[junit4] > at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.lambda$httpUriRequest$0(HttpSolrClient.java:299)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient$$Lambda$93/995676679.call(Unknown Source)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1371611390.run(Unknown Source)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([9CBE136414B5BF3A]:0)
[junit4] Completed [459/615 (2!)] on J0 in 638.62s, 1 test, 2 errors <<< FAILURES!
[...truncated 513 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:740: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:684: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/solr/common-build.xml:531: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:1427: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.x/lucene/common-build.xml:984: There were test failures: 615 suites (10 ignored), 2591 tests, 4 suite-level errors, 1925 ignored (64 assumptions) [seed: 9CBE136414B5BF3A]
Total time: 49 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any