You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/08/17 05:06:44 UTC

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-15-ea+35) - Build # 27690 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/27690/
Java: 64bit/jdk-15-ea+35 -XX:+UseCompressedOops -XX:+UseSerialGC

12 tests failed.
FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35321_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640767777729004"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:35321/solr",           "node_name":"127.0.0.1:35321_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640767777749413"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35321_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/22)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640767777729004"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:35321/solr",
          "node_name":"127.0.0.1:35321_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640767777749413"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:44023_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640783167527356"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:44023/solr",           "node_name":"127.0.0.1:44023_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640783167548476"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:44023_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/22)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640783167527356"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:44023/solr",
          "node_name":"127.0.0.1:44023_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640783167548476"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:37453_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640787690493699"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:37453/solr",           "node_name":"127.0.0.1:37453_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640787690514199"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:37453_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/22)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640787690493699"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:37453/solr",
          "node_name":"127.0.0.1:37453_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640787690514199"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:34167_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640787636189431"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:34167/solr",           "node_name":"127.0.0.1:34167_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640787636210922"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:34167_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/22)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640787636189431"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:34167/solr",
          "node_name":"127.0.0.1:34167_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640787636210922"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:39177_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/24)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640746012957141"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:39177/solr",           "node_name":"127.0.0.1:39177_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640746012989683"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:39177_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/24)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640746012957141"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:39177/solr",
          "node_name":"127.0.0.1:39177_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640746012989683"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35095_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(backuprestore_testok_restored/23)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"backuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"backuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"backuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"backuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597639506069965114"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"backuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"backuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"backuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"backuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"https://127.0.0.1:35095/solr",           "node_name":"127.0.0.1:35095_solr",           "state":"active",           "collection":"backuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597639506069980844"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35095_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(backuprestore_testok_restored/23)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"backuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"backuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"backuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"backuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597639506069965114"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"backuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"backuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"backuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"backuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"https://127.0.0.1:35095/solr",
          "node_name":"127.0.0.1:35095_solr",
          "state":"active",
          "collection":"backuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597639506069980844"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:35367_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(hdfsbackuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640764541542291"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"hdfsbackuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node12":{           "core":"hdfsbackuprestore_testok_restored_shard1_replica_n11",           "core_node_name":"core_node12",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard1",           "type":"NRT",           "force_set_state":"false"},         "core_node14":{           "core":"hdfsbackuprestore_testok_restored_shard1_replica_t13",           "core_node_name":"core_node14",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard1",           "type":"TLOG",           "force_set_state":"false"},         "core_node16":{           "core":"hdfsbackuprestore_testok_restored_shard1_replica_p15",           "core_node_name":"core_node16",           "base_url":"http://127.0.0.1:35367/solr",           "node_name":"127.0.0.1:35367_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard1",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640764541565986"}},   "router":{     "name":"implicit",     "field":"shard_s"},   "nrtReplicas":2,   "tlogReplicas":1}

Stack Trace:
java.lang.AssertionError: Node 127.0.0.1:35367_solr has 8 replicas. Expected num replicas : 4. state: 
DocCollection(hdfsbackuprestore_testok_restored/22)={
  "pullReplicas":1,
  "replicationFactor":2,
  "shards":{
    "shard2":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node2":{
          "core":"hdfsbackuprestore_testok_restored_shard2_replica_n1",
          "leader":"true",
          "core_node_name":"core_node2",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node6":{
          "core":"hdfsbackuprestore_testok_restored_shard2_replica_n5",
          "core_node_name":"core_node6",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard2",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node8":{
          "core":"hdfsbackuprestore_testok_restored_shard2_replica_t7",
          "core_node_name":"core_node8",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard2",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node10":{
          "core":"hdfsbackuprestore_testok_restored_shard2_replica_p9",
          "core_node_name":"core_node10",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard2",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640764541542291"},
    "shard1":{
      "range":null,
      "state":"active",
      "replicas":{
        "core_node4":{
          "core":"hdfsbackuprestore_testok_restored_shard1_replica_n3",
          "leader":"true",
          "core_node_name":"core_node4",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node12":{
          "core":"hdfsbackuprestore_testok_restored_shard1_replica_n11",
          "core_node_name":"core_node12",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard1",
          "type":"NRT",
          "force_set_state":"false"},
        "core_node14":{
          "core":"hdfsbackuprestore_testok_restored_shard1_replica_t13",
          "core_node_name":"core_node14",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard1",
          "type":"TLOG",
          "force_set_state":"false"},
        "core_node16":{
          "core":"hdfsbackuprestore_testok_restored_shard1_replica_p15",
          "core_node_name":"core_node16",
          "base_url":"http://127.0.0.1:35367/solr",
          "node_name":"127.0.0.1:35367_solr",
          "state":"active",
          "collection":"hdfsbackuprestore_testok_restored",
          "shard":"shard1",
          "type":"PULL",
          "force_set_state":"false"}},
      "stateTimestamp":"1597640764541565986"}},
  "router":{
    "name":"implicit",
    "field":"shard_s"},
  "nrtReplicas":2,
  "tlogReplicas":1}
	at __randomizedtesting.SeedInfo.seed([C0E3EE46BBB498B2:48B7D19C1548F54A]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:374)
	at java.base/java.util.HashMap.forEach(HashMap.java:1425)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:373)
	at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:155)
	at org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test(TestHdfsCloudBackupRestore.java:225)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test

Error Message:
Node 127.0.0.1:36771_solr has 8 replicas. Expected num replicas : 4. state:  DocCollection(hdfsbackuprestore_testok_restored/22)={   "pullReplicas":1,   "replicationFactor":2,   "shards":{     "shard2":{       "range":null,       "state":"active",       "replicas":{         "core_node2":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_n1",           "leader":"true",           "core_node_name":"core_node2",           "base_url":"http://127.0.0.1:36771/solr",           "node_name":"127.0.0.1:36771_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node6":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_n5",           "core_node_name":"core_node6",           "base_url":"http://127.0.0.1:36771/solr",           "node_name":"127.0.0.1:36771_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"NRT",           "force_set_state":"false"},         "core_node8":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_t7",           "core_node_name":"core_node8",           "base_url":"http://127.0.0.1:36771/solr",           "node_name":"127.0.0.1:36771_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"TLOG",           "force_set_state":"false"},         "core_node10":{           "core":"hdfsbackuprestore_testok_restored_shard2_replica_p9",           "core_node_name":"core_node10",           "base_url":"http://127.0.0.1:36771/solr",           "node_name":"127.0.0.1:36771_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard2",           "type":"PULL",           "force_set_state":"false"}},       "stateTimestamp":"1597640764812536651"},     "shard1":{       "range":null,       "state":"active",       "replicas":{         "core_node4":{           "core":"hdfsbackuprestore_testok_restored_shard1_replica_n3",           "leader":"true",           "core_node_name":"core_node4",           "base_url":"http://127.0.0.1:36771/solr",           "node_name":"127.0.0.1:36771_solr",           "state":"active",           "collection":"hdfsbackuprestore_testok_restored",           "shard":"shard1",   

[...truncated too long message...]

rMetricManager Closing metric reporters for registry=solr.collection.backuprestore_testok_restored.shard1.leader tag=SolrCore@6fddecb4
   [junit4]   2> 71222 INFO  (coreCloseExecutor-442-thread-13) [n:127.0.0.1:34167_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 71222 INFO  (coreCloseExecutor-442-thread-11) [n:127.0.0.1:34167_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 71222 INFO  (coreCloseExecutor-442-thread-14) [n:127.0.0.1:34167_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 71222 INFO  (coreCloseExecutor-442-thread-12) [n:127.0.0.1:34167_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 71228 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 71229 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@57248e02: rootName = solr_34167, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@3db0f697
   [junit4]   2> 71233 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 71233 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@212759ec: rootName = solr_34167, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@2ed4a15
   [junit4]   2> 71235 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 71236 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3900724a: rootName = solr_34167, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@680ccfaf
   [junit4]   2> 71236 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 71237 INFO  (closeThreadPool-447-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72213433201459205-127.0.0.1:34167_solr-n_0000000000) closing
   [junit4]   2> 71237 INFO  (OverseerStateUpdate-72213433201459205-127.0.0.1:34167_solr-n_0000000000) [n:127.0.0.1:34167_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34167_solr
   [junit4]   2> 71239 INFO  (closeThreadPool-447-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72213433201459205-127.0.0.1:34167_solr-n_0000000000) closing
   [junit4]   2> 71290 INFO  (jetty-closer-435-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@4d2cc9be{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
   [junit4]   2> 71290 INFO  (jetty-closer-435-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@44087583{/solr,null,UNAVAILABLE}
   [junit4]   2> 71290 INFO  (jetty-closer-435-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 71388 INFO  (jetty-closer-435-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72213433201459205-127.0.0.1:34167_solr-n_0000000000) closing
   [junit4]   2> 71394 INFO  (jetty-closer-435-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@154dcd9a{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
   [junit4]   2> 71394 INFO  (jetty-closer-435-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5f455e5a{/solr,null,UNAVAILABLE}
   [junit4]   2> 71394 INFO  (jetty-closer-435-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 71395 INFO  (SUITE-TestLocalFSCloudBackupRestore-seed#[C0E3EE46BBB498B2]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 71597 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	10	/solr/collections/backuprestore_testok/terms/shard1
   [junit4]   2> 	9	/solr/collections/backuprestore_testok_restored/terms/shard2
   [junit4]   2> 	9	/solr/collections/backuprestore_testok_restored/terms/shard1
   [junit4]   2> 	7	/solr/collections/backuprestore_testfailure/terms/shard1
   [junit4]   2> 	7	/solr/collections/backuprestore_testfailure/terms/shard2
   [junit4]   2> 	7	/solr/collections/backuprestore_testok/terms/shard2
   [junit4]   2> 	3	/solr/aliases.json
   [junit4]   2> 	2	/solr/collections/backuprestore_testfailure/collectionprops.json
   [junit4]   2> 	2	/solr/packages.json
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 	2	/solr/collections/backuprestore_testok/collectionprops.json
   [junit4]   2> 	2	/solr/configs/conf1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	92	/solr/collections/backuprestore_testok_restored/state.json
   [junit4]   2> 	43	/solr/collections/backuprestore_testok/state.json
   [junit4]   2> 	36	/solr/collections/backuprestore_testfailure/state.json
   [junit4]   2> 	3	/solr/clusterprops.json
   [junit4]   2> 	3	/solr/collections/backuprestore_testfailure_restored/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	130	/solr/overseer/queue
   [junit4]   2> 	18	/solr/collections
   [junit4]   2> 	16	/solr/overseer/collection-queue-work
   [junit4]   2> 	9	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 71600 INFO  (SUITE-TestLocalFSCloudBackupRestore-seed#[C0E3EE46BBB498B2]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:39177
   [junit4]   2> 71600 INFO  (SUITE-TestLocalFSCloudBackupRestore-seed#[C0E3EE46BBB498B2]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:39177
   [junit4]   2> 71600 INFO  (SUITE-TestLocalFSCloudBackupRestore-seed#[C0E3EE46BBB498B2]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 39177
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J3/temp/solr.cloud.api.collections.TestLocalFSCloudBackupRestore_C0E3EE46BBB498B2-001
   [junit4]   2> Aug 17, 2020 5:06:35 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene86): {shard_s=Lucene84, _root_=Lucene84, id=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1616, maxMBSortInHeap=5.363107240034312, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=ru-KZ, timezone=America/Kralendijk
   [junit4]   2> NOTE: Linux 5.4.0-40-generic amd64/Oracle Corporation 15 (64-bit)/cpus=16,threads=1,free=362120528,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestHdfsCloudBackupRestore, TestLocalFSCloudBackupRestore]
   [junit4] Completed [10/10 (10!)] on J3 in 22.20s, 2 tests, 1 failure <<< FAILURES!

[...truncated 23 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1521: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1048: There were test failures: 10 suites, 20 tests, 10 failures [seed: C0E3EE46BBB498B2]

Total time: 1 minute 14 seconds

[repro] Setting last failure code to 256

[repro] Failures w/original seeds:
[repro]   5/5 failed: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[repro]   5/5 failed: org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Archiving artifacts
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/home/jenkins/workspace/Lucene-Solr-master-Linux')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-Linux #27689' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-Linux #27690'.
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Recording test results
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-15-ea+35) - Build # 27691 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/27691/
Java: 64bit/jdk-15-ea+35 -XX:-UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestBulkSchemaConcurrent

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)   expected null, but was:<ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  >

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

 expected null, but was:<ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

>
	at __randomizedtesting.SeedInfo.seed([E4051C84557FA603]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotNull(Assert.java:755)
	at org.junit.Assert.assertNull(Assert.java:737)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:327)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:905)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestBulkSchemaConcurrent

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)   expected null, but was:<ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)  at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)  at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)  at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)  at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)  at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)  at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)  at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)  at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)  at org.eclipse.jetty.server.Server.handle(Server.java:500)  at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)  at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)  at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)  at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)  at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)  at java.base/java.lang.Thread.run(Thread.java:832)  >

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

 expected null, but was:<ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:98)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:817)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
	at org.eclipse.jetty.server.Server.handle(Server.java:500)
	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
	at java.base/java.lang.Thread.run(Thread.java:832)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
	at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:557)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1001)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolr

[...truncated too long message...]

> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
   [junit4]    > 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
   [junit4]    > 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
   [junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
   [junit4]    > 	at org.eclipse.jetty.server.Server.handle(Server.java:500)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:832)
   [junit4]    > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
   [junit4]    > 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:354)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:406)
   [junit4]    > 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:785)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1005)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:925)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1357)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1243)
   [junit4]    > 	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:95)
   [junit4]    > 	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
   [junit4]    > 	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
   [junit4]    > 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
   [junit4]    > 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)
   [junit4]    > 	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:854)
   [junit4]    > 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:818)
   [junit4]    > 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:566)
   [junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:437)
   [junit4]    > 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:351)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
   [junit4]    > 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
   [junit4]    > 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
   [junit4]    > 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
   [junit4]    > 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
   [junit4]    > 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
   [junit4]    > 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
   [junit4]    > 	at org.eclipse.jetty.server.Server.handle(Server.java:500)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
   [junit4]    > 	at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:335)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
   [junit4]    > 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:832)
   [junit4]    > >
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([E4051C84557FA603]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:327)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:832)
   [junit4] Completed [5/5 (3!)] on J0 in 87.45s, 1 test, 1 failure <<< FAILURES!

[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1521: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1048: There were test failures: 5 suites, 5 tests, 3 suite-level errors [seed: E4051C84557FA603]

Total time: 1 minute 29 seconds

[repro] Setting last failure code to 256

[repro] Failures w/original seeds:
[repro]   3/5 failed: org.apache.solr.schema.TestBulkSchemaConcurrent
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Archiving artifacts
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/home/jenkins/workspace/Lucene-Solr-master-Linux')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-Linux #27690' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-Linux #27691'.
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Recording test results
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2