You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/05/26 04:45:53 UTC
[JENKINS-Experimental-GC] Lucene-Solr-master-MacOSX (64bit/jdk-14)
- Build # 6071 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6071/
Java: 64bit/jdk-14 -XX:+UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseZGC
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
Error Message:
ObjectTracker found 7 object(s) that were not released!!! [MMapDirectory, InternalHttpClient, MMapDirectory, MMapDirectory, SolrCore, MMapDirectory, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:519) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:422) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1208) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:418) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:250) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:680) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:391) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:767) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:539) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:799) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1239) at org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109) at org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1330) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:711) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) expected null, but was:<ObjectTracker found 7 object(s) that were not released!!! [MMapDirectory, InternalHttpClient, MMapDirectory, MMapDirectory, SolrCore, MMapDirectory, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:519) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:422) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1208) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:418) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:250) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:680) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:391) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:767) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:539) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:799) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1239) at org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109) at org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1330) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:711) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) at java.base/java.lang.Thread.run(Thread.java:832) >
Stack Trace:
java.lang.AssertionError: ObjectTracker found 7 object(s) that were not released!!! [MMapDirectory, InternalHttpClient, MMapDirectory, MMapDirectory, SolrCore, MMapDirectory, InternalHttpClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:519)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:422)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1208)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:418)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:250)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:680)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:391)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:767)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:539)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:799)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1239)
at org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
at org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1330)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:711)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
expected null, but was:<ObjectTracker found 7 object(s) that were not released!!! [MMapDirectory, InternalHttpClient, MMapDirectory, MMapDirectory, SolrCore, MMapDirectory, InternalHttpClient]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:519)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:422)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1208)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:418)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:250)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:680)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:391)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:767)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:539)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:799)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1239)
at org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
at org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1330)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:711)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
>
at __randomizedtesting.SeedInfo.seed([4E6B6D21EFAC642E]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotNull(Assert.java:755)
at org.junit.Assert.assertNull(Assert.java:737)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:334)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:905)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
at java.base/java.lang.Thread.run(Thread.java:832)
Build Log:
[...truncated 14718 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[junit4] 2> 1496634 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/server/solr/configsets/_default/conf'
[junit4] 2> 1496634 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/data-dir-99-001
[junit4] 2> 1496635 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=78 numCloses=78
[junit4] 2> 1496635 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1496636 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1496636 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1496716 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1496718 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14+36
[junit4] 2> 1496720 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1496720 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1496720 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1496720 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@184a4ef5{static,/static,jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 1496837 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@730d3679{hdfs,/,file:///Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/jetty-localhost-51240-hadoop-hdfs-3_2_0-tests_jar-_-any-12736850029760366342.dir/webapp/,AVAILABLE}{jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
[junit4] 2> 1496837 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6a21ffc2{HTTP/1.1, (http/1.1)}{localhost:51240}
[junit4] 2> 1496837 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.Server Started @1496934ms
[junit4] 2> 1496906 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1496907 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14+36
[junit4] 2> 1496908 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1496908 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1496908 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1496909 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c8eeaeb{static,/static,jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 1497033 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@6111aabd{datanode,/,file:///Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/jetty-localhost-51243-hadoop-hdfs-3_2_0-tests_jar-_-any-15695616775048934073.dir/webapp/,AVAILABLE}{jar:file:/Users/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
[junit4] 2> 1497034 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@30b3cfd9{HTTP/1.1, (http/1.1)}{localhost:51243}
[junit4] 2> 1497034 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.e.j.s.Server Started @1497131ms
[junit4] 2> 1497172 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xff2fab9937ba2358: Processing first storage report for DS-d75b1e55-53de-4c22-8e46-0a15b2d62e1c from datanode 78f765b6-34e2-443d-a901-3dc8df1862d8
[junit4] 2> 1497173 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xff2fab9937ba2358: from storage DS-d75b1e55-53de-4c22-8e46-0a15b2d62e1c node DatanodeRegistration(127.0.0.1:51242, datanodeUuid=78f765b6-34e2-443d-a901-3dc8df1862d8, infoPort=51244, infoSecurePort=0, ipcPort=51245, storageInfo=lv=-57;cid=testClusterID;nsid=1767066769;c=1590466313685), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0
[junit4] 2> 1497174 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xff2fab9937ba2358: Processing first storage report for DS-f61c8459-e26d-4ccf-97ea-afe626f0a005 from datanode 78f765b6-34e2-443d-a901-3dc8df1862d8
[junit4] 2> 1497174 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xff2fab9937ba2358: from storage DS-f61c8459-e26d-4ccf-97ea-afe626f0a005 node DatanodeRegistration(127.0.0.1:51242, datanodeUuid=78f765b6-34e2-443d-a901-3dc8df1862d8, infoPort=51244, infoSecurePort=0, ipcPort=51245, storageInfo=lv=-57;cid=testClusterID;nsid=1767066769;c=1590466313685), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0
[junit4] 2> 1497292 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002
[junit4] 2> 1497292 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1497293 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
[junit4] 2> 1497293 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1497395 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port: 51250
[junit4] 2> 1497395 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:51250
[junit4] 2> 1497395 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:51250
[junit4] 2> 1497395 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 51250
[junit4] 2> 1497416 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497423 INFO (zkConnectionManagerCallback-12914-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497423 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497434 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497441 INFO (zkConnectionManagerCallback-12916-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497441 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497442 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497448 INFO (zkConnectionManagerCallback-12918-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497448 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14+36
[junit4] 2> 1497579 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.Server jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 14+36
[junit4] 2> 1497583 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1497583 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1497584 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1497585 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1497585 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1497585 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1497586 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35b8771d{/solr,null,AVAILABLE}
[junit4] 2> 1497586 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@424ca87b{/solr,null,AVAILABLE}
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14c9a025{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:51259}
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.e.j.s.Server Started @1497685ms
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51259}
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6cf62166{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:51258}
[junit4] 2> 1497589 ERROR (jetty-launcher-12919-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-2) [ ] o.e.j.s.Server Started @1497686ms
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51258}
[junit4] 2> 1497589 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-05-26T04:11:54.621015Z
[junit4] 2> 1497590 ERROR (jetty-launcher-12919-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 9.0.0
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr
[junit4] 2> 1497590 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2020-05-26T04:11:54.621316Z
[junit4] 2> 1497591 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497591 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497598 INFO (zkConnectionManagerCallback-12921-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497598 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497599 INFO (zkConnectionManagerCallback-12923-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497599 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497602 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1497602 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1497609 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1497610 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1497860 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1497868 WARN (jetty-launcher-12919-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@132dfbda[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497868 WARN (jetty-launcher-12919-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@132dfbda[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497873 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1497873 WARN (jetty-launcher-12919-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2166817c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497873 WARN (jetty-launcher-12919-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2166817c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497874 INFO (jetty-launcher-12919-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51250/solr
[junit4] 2> 1497874 WARN (jetty-launcher-12919-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@48967dae[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497874 WARN (jetty-launcher-12919-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@48967dae[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497880 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497888 INFO (zkConnectionManagerCallback-12941-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497888 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1497890 WARN (jetty-launcher-12919-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4050339c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497890 WARN (jetty-launcher-12919-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4050339c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1497891 INFO (jetty-launcher-12919-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51250/solr
[junit4] 2> 1497893 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1497897 INFO (zkConnectionManagerCallback-12945-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1497897 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1498496 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51259_solr
[junit4] 2> 1498500 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.Overseer Overseer (id=72106167229480965-127.0.0.1:51259_solr-n_0000000000) starting
[junit4] 2> 1498559 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:51259_solr
[junit4] 2> 1498561 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51259_solr
[junit4] 2> 1498572 INFO (zkCallback-12940-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1498582 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 1498585 WARN (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 1498585 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup, solr.hdfs.home=hdfs://localhost:51241/solr, solr.hdfs.confdir=}}
[junit4] 2> 1498585 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = poisioned,class = org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository,attributes = {default=true, name=poisioned, class=org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository},}
[junit4] 2> 1498585 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = poisioned,class = org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository,attributes = {default=true, name=poisioned, class=org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository},}
[junit4] 2> 1498615 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1498653 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498660 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498660 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498661 INFO (jetty-launcher-12919-thread-1) [n:127.0.0.1:51259_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node1/.
[junit4] 2> 1498831 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1498870 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:51258_solr as DOWN
[junit4] 2> 1498873 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
[junit4] 2> 1498873 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51258_solr
[junit4] 2> 1498878 INFO (zkCallback-12940-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1498879 INFO (zkCallback-12944-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1498882 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 1498882 WARN (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 1498882 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup, solr.hdfs.home=hdfs://localhost:51241/solr, solr.hdfs.confdir=}}
[junit4] 2> 1498882 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = poisioned,class = org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository,attributes = {default=true, name=poisioned, class=org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository},}
[junit4] 2> 1498882 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = poisioned,class = org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository,attributes = {default=true, name=poisioned, class=org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore$PoinsionedRepository},}
[junit4] 2> 1498899 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1498924 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498939 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498939 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1498940 INFO (jetty-launcher-12919-thread-2) [n:127.0.0.1:51258_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node2/.
[junit4] 2> 1498947 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
[junit4] 2> 1498948 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 1498950 INFO (zkConnectionManagerCallback-12970-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1498950 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 1498953 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1498955 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[4E6B6D21EFAC642E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51250/solr ready
[junit4] 2> 1499059 INFO (TEST-TestHdfsCloudBackupRestore.testRestoreFailure-seed#[4E6B6D21EFAC642E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRestoreFailure
[junit4] 2> 1499064 INFO (qtp1647432528-21057) [n:127.0.0.1:51258_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&collection.configName=conf1&maxShardsPerNode=3&name=hdfsbackuprestore_testfailure&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1499070 INFO (OverseerThreadFactory-12954-thread-1-processing-n:127.0.0.1:51259_solr) [n:127.0.0.1:51259_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection hdfsbackuprestore_testfailure
[junit4] 2> 1499187 WARN (OverseerThreadFactory-12954-thread-1-processing-n:127.0.0.1:51259_solr) [n:127.0.0.1:51259_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 3 on collection hdfsbackuprestore_testfailure is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 1499191 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
[junit4] 2> 1499193 INFO (qtp437148948-21053) [n:127.0.0.1:51259_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 1000 transient cores
[junit4] 2> 1499193 INFO (qtp437148948-21053) [n:127.0.0.1:51259_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
[junit4] 2> 1499195 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
[junit4] 2> 1499195 INFO (qtp437148948-21056) [n:127.0.0.1:51259_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CONTAINER.cores&wt=javabin&version=2&group=solr.node} status=0 QTime=0
[junit4] 2> 1499209 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51259/solr",
[junit4] 2> "node_name":"127.0.0.1:51259_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499214 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51258/solr",
[junit4] 2> "node_name":"127.0.0.1:51258_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499219 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard1_replica_p4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51258/solr",
[junit4] 2> "node_name":"127.0.0.1:51258_solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499225 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51259/solr",
[junit4] 2> "node_name":"127.0.0.1:51259_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499232 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard2_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51258/solr",
[junit4] 2> "node_name":"127.0.0.1:51258_solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499248 INFO (OverseerStateUpdate-72106167229480965-127.0.0.1:51259_solr-n_0000000000) [n:127.0.0.1:51259_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore_testfailure",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_testfailure_shard2_replica_p10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51259/solr",
[junit4] 2> "node_name":"127.0.0.1:51259_solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1499449 INFO (qtp1647432528-21060) [n:127.0.0.1:51258_solr x:hdfsbackuprestore_testfailure_shard1_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard1_replica_n2&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1499449 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard1_replica_n1&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1499451 INFO (qtp437148948-21053) [n:127.0.0.1:51259_solr x:hdfsbackuprestore_testfailure_shard2_replica_n6 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard2_replica_n6&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1499451 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard2_replica_n8&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1499452 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard1_replica_p4&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 1499452 INFO (qtp437148948-21051) [n:127.0.0.1:51259_solr x:hdfsbackuprestore_testfailure_shard2_replica_p10 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf1&newCollection=true&name=hdfsbackuprestore_testfailure_shard2_replica_p10&action=CREATE&numShards=2&collection=hdfsbackuprestore_testfailure&shard=shard2&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 1499495 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499498 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499508 INFO (qtp1647432528-21060) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node5 x:hdfsbackuprestore_testfailure_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499508 INFO (qtp437148948-21053) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node9 x:hdfsbackuprestore_testfailure_shard2_replica_n6 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499514 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499520 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.s.IndexSchema Schema name=minimal
[junit4] 2> 1499526 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
[junit4] 2> 1499985 INFO (qtp437148948-21051) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node12 x:hdfsbackuprestore_testfailure_shard2_replica_p10 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 1499985 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1499986 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_testfailure_shard1_replica_p4' using configuration from configset conf1, trusted=true
[junit4] 2> 1499987 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1499987 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_testfailure_shard1_replica_n1' using configuration from configset conf1, trusted=true
[junit4] 2> 1499987 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore_testfailure.shard1.replica_n1' (registry 'solr.core.hdfsbackuprestore_testfailure.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1499987 INFO (qtp437148948-21058) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node3 x:hdfsbackuprestore_testfailure_shard1_replica_n1 ] o.a.s.c.SolrCore [[hdfsbackuprestore_testfailure_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node1/./hdfsbackuprestore_testfailure_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node1/./hdfsbackuprestore_testfailure_shard1_replica_n1/data/]
[junit4] 2> 1499988 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore_testfailure.shard1.replica_p4' (registry 'solr.core.hdfsbackuprestore_testfailure.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1499988 INFO (qtp1647432528-21055) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node7 x:hdfsbackuprestore_testfailure_shard1_replica_p4 ] o.a.s.c.SolrCore [[hdfsbackuprestore_testfailure_shard1_replica_p4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node2/./hdfsbackuprestore_testfailure_shard1_replica_p4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/build/solr-core/test/J4/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_4E6B6D21EFAC642E-001/tempDir-002/node2/./hdfsbackuprestore_testfailure_shard1_replica_p4/data/]
[junit4] 2> 1499996 INFO (qtp437148948-21053) [n:127.0.0.1:51259_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node9 x:hdfsbackuprestore_testfailure_shard2_replica_n6 ] o.a.s.s.IndexSchema Schema name=minimal
[junit4] 2> 1499996 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.s.IndexSchema Schema name=minimal
[junit4] 2> 1499997 INFO (qtp1647432528-21060) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard1 r:core_node5 x:hdfsbackuprestore_testfailure_shard1_replica_n2 ] o.a.s.s.IndexSchema Schema name=minimal
[junit4] 2> 1500007 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1500007 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_testfailure_shard2_replica_n8' using configuration from configset conf1, trusted=true
[junit4] 2> 1500007 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard2_replica_n8 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore_testfailure.shard2.replica_n8' (registry 'solr.core.hdfsbackuprestore_testfailure.shard2.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@222bc8d4
[junit4] 2> 1500007 INFO (qtp1647432528-21061) [n:127.0.0.1:51258_solr c:hdfsbackuprestore_testfailure s:shard2 r:core_node11 x:hdfsbackuprestore_testfailure_shard
[...truncated too long message...]
track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
[junit4] > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:832)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
[junit4] > at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:539)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
[junit4] > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:832)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:339)
[junit4] > at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:348)
[junit4] > at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
[junit4] > at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
[junit4] > at org.apache.solr.handler.ReplicationHandler.inform(ReplicationHandler.java:1239)
[junit4] > at org.apache.solr.cloud.ReplicateFromLeader.startReplication(ReplicateFromLeader.java:109)
[junit4] > at org.apache.solr.cloud.ZkController.startReplicationFromLeader(ZkController.java:1330)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:711)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:332)
[junit4] > at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:315)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
[junit4] > at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[junit4] > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:832)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
[junit4] > at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
[junit4] > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:799)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:985)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:903)
[junit4] > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1307)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1216)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:93)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:362)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:189)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:832)
[junit4] > >
[junit4] > at __randomizedtesting.SeedInfo.seed([4E6B6D21EFAC642E]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:334)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:832)
[junit4] Completed [5/5 (1!)] on J1 in 146.51s, 2 tests, 1 failure <<< FAILURES!
[...truncated 13 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:1521: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/lucene/common-build.xml:1048: There were test failures: 5 suites, 10 tests, 1 suite-level error [seed: 4E6B6D21EFAC642E]
Total time: 2 minutes 31 seconds
[repro] Setting last failure code to 256
[repro] Failures w/original seeds:
[repro] 1/5 failed: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
No credentials specified
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=675956c0041b18d48a7c059ea458c49f5310d74a, workspace=/Users/jenkins/workspace/Lucene-Solr-master-MacOSX
[WARNINGS] Computing warning deltas based on reference build #6070
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2