SOLR Unable to Create CORE - SOLR Version 8.9

[Fiz N <fi...@gmail.com>]

Hi  All,

While creating any new collection we are getting SOLR Core Initialization
failure on Admin UI.



We see below errors in the Solr logs,

(Error CREATEing SolrCore 'snlp_is_dummy_test_shard4_replica_n20': Unable
to create core [snlp_is_dummy_test_shard4_replica_n20] Caused by:

 KeeperErrorCode = NoNode for /collections/snlp_is_dummy_test),



(Caused by: org.apache.solr.common.SolrException: Failed to load config
name for collection:snlp_is_dummy_test due to:
    at
org.apache.solr.rest.ManagedResourceStorage.newStorageIO(ManagedResourceStorage.java:97)
    at org.apache.solr.core.SolrCore.initRestManager(SolrCore.java:2971)
    at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1030)
    ... 52 more)

There are no errors or warnings in the ZK logs.

*Other notable information*

   - Physical Memory is above 99% in Prod. Thou it is the same in Test
   servers too, the issue is only in Prod; while Test seems to work fine.



   - *Memory usage in Prod*


   - Prod collection count -58
      - physical memory- out of 123GB 122.38 GB consumed i.e(99.5%)
      - JVM-out of 32GB 23.44 GB consumed i.e(73.3%)



*Test **for comparison (as it works here)*

Test collection count-79

physical memory- out of 123GB 121.46 GB consumed i.e(98.7%)

JVM-out of 32GB 10.80 GB consumed i.e(33.7%)


Regards

Fiz.