You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by "Oakley, Craig (NIH/NLM/NCBI) [C] via users" <us...@solr.apache.org> on 2025/12/04 18:18:24 UTC
RE: [EXTERNAL] Re: ZkCredentialsInjector issue
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk" (as previously mentioned)
SOLR_ZK_CREDS_AND_ACLS is
-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector -DzkDigestCredentialsFile=[the redacted filename]
From: lamine lamine <ya...@yahoo.fr>
Sent: Thursday, December 4, 2025 12:27 PM
To: users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: [EXTERNAL] Re: ZkCredentialsInjector issue
The error says
: access denied ("java.io.FilePermission" "[file name redacted]" "read")
Can you check the permissions?
Can you as well share the config you're using for SOLR_ZK_CREDS_AND_ACLS
?
On Thu, Dec 4, 2025 at 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
For many years, we have been passing Zookeeper credentials through command line arguments (https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-system-properties): but it would be nice to hide the passwords away in a credentials file as described in https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-a-file
This succeeds when using "bin/solr zk", but not for the Solr instance itself. The solr.log file complains
2025-12-04 11:10:06.851 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsInjector: org.apache.solr.common.cloud.VMParamsZkCredentialsInjector
2025-12-04 11:10:06.852 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param ZkCredentialsInjector does not point to a class implementing ZkCredentialsInjector and with a non-arg constructor
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 61 more
2025-12-04 11:10:06.854 WARN (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
...
2025-12-04 11:10:06.970 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs
2025-12-04 11:10:06.982 ERROR (main) [ ] o.a.s.c.SolrCore null
org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.common.cloud.VMParamsZkCredentialsInjector'
at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:649) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
... 55 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 55 more
2025-12-04 11:10:06.990 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer is either not initialized or shutting down.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.c.SolrCore null
javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.CoreContainerProvider.waitForCoreContainer(CoreContainerProvider.java:154) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
2025-12-04 11:10:07.002 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@578524c3{solr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp}<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d>>
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk"
Any suggestions what to try?
Thanks
Craig Oakley
Contract Worker (Other)
National Library of Medicine
National Institutes of Health
Building 45, room 6AS37D-57
301-496-6175
oakleycd@ncbi.nlm.nih.gov<ma...@ncbi.nlm.nih.gov>
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Posted by lamine lamine via users <us...@solr.apache.org>.
I confirm, zk creds injector needs be added to solr.xml as this is a new feature that old solr versions do not support.
We should add a note to the documention about upgrades
On Tue, Dec 16, 2025 at 11:02 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users<us...@solr.apache.org> wrote: I wanted to add one more note to this thread: another piece of the puzzle in case someone trying to resolve the same issue happens upon this thread.
One may need to add a line to solr.xml
The solr instance where I first tried to resolve this issue happened to have this line: but very few of my other solr.xml files included the line. Reproducing the exact same set of changes on other instances initially proved insufficient: but I eventually found that what was needed was to add this line within the <solrcloud> clause of solr.xml
<str name="zkCredentialsInjector">${zkCredentialsInjector:org.apache.solr.common.cloud.DefaultZkCredentialsInjector}</str>
I guess my other solr.xml files had originally come from earlier versions of Solr without that line
-----Original Message-----
From: Oakley, Craig (NIH/NLM/NCBI) [C] via users <us...@solr.apache.org>
Sent: Thursday, December 4, 2025 11:05 PM
To: lamine lamine <ya...@yahoo.fr>; users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: RE: [EXTERNAL] Re: ZkCredentialsInjector issue
> Is your fille in.a location allowed by solr's security policy?
That was the relevant question
I had it in a directory next door to $SOLR_HOME (on a protected file system): a sibling directory from which we read our SOLR_ZK_CREDS_AND_ACLS: but of course we read SOLR_ZK_CREDS_AND_ACLS before we get into Java: and apparently the failure to read is coming from Java
I have moved the properties file into $SOLR_HOME itself; and now it works
Thanks
From: lamine lamine <ya...@yahoo.fr>
Sent: Thursday, December 4, 2025 8:14 PM
To: users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Is your fille in.a location allowed by solr's security policy?
Can you put it in "server/etc" like in the example?
DzkDigestCredentialsFile=
SOLR_HOME_DIR/server/etc/zookeepercredentials.properties"
On Thu, Dec 4, 2025 at 12:19 PM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk" (as previously mentioned)
SOLR_ZK_CREDS_AND_ACLS is
-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector -DzkDigestCredentialsFile=[the redacted filename]
From: lamine lamine <ya...@yahoo.fr>>
Sent: Thursday, December 4, 2025 12:27 PM
To: users@solr.apache.org<ma...@solr.apache.org>
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>>
Subject: [EXTERNAL] Re: ZkCredentialsInjector issue
The error says
: access denied ("java.io.FilePermission" "[file name redacted]" "read")
Can you check the permissions?
Can you as well share the config you're using for SOLR_ZK_CREDS_AND_ACLS
?
On Thu, Dec 4, 2025 at 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
For many years, we have been passing Zookeeper credentials through command line arguments (https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-system-properties): but it would be nice to hide the passwords away in a credentials file as described in https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-a-file
This succeeds when using "bin/solr zk", but not for the Solr instance itself. The solr.log file complains
2025-12-04 11:10:06.851 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsInjector: org.apache.solr.common.cloud.VMParamsZkCredentialsInjector
2025-12-04 11:10:06.852 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param ZkCredentialsInjector does not point to a class implementing ZkCredentialsInjector and with a non-arg constructor
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 61 more
2025-12-04 11:10:06.854 WARN (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
...
2025-12-04 11:10:06.970 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs
2025-12-04 11:10:06.982 ERROR (main) [ ] o.a.s.c.SolrCore null
org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.common.cloud.VMParamsZkCredentialsInjector'
at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:649) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
... 55 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 55 more
2025-12-04 11:10:06.990 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer is either not initialized or shutting down.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.c.SolrCore null
javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.CoreContainerProvider.waitForCoreContainer(CoreContainerProvider.java:154) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
2025-12-04 11:10:07.002 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@578524c3{solr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp}<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d>>>
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk"
Any suggestions what to try?
Thanks
Craig Oakley
Contract Worker (Other)
National Library of Medicine
National Institutes of Health
Building 45, room 6AS37D-57
301-496-6175
oakleycd@ncbi.nlm.nih.gov<ma...@ncbi.nlm.nih.gov>
<ma...@ncbi.nlm.nih.gov>
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Posted by "Oakley, Craig (NIH/NLM/NCBI) [C] via users" <us...@solr.apache.org>.
I wanted to add one more note to this thread: another piece of the puzzle in case someone trying to resolve the same issue happens upon this thread.
One may need to add a line to solr.xml
The solr instance where I first tried to resolve this issue happened to have this line: but very few of my other solr.xml files included the line. Reproducing the exact same set of changes on other instances initially proved insufficient: but I eventually found that what was needed was to add this line within the <solrcloud> clause of solr.xml
<str name="zkCredentialsInjector">${zkCredentialsInjector:org.apache.solr.common.cloud.DefaultZkCredentialsInjector}</str>
I guess my other solr.xml files had originally come from earlier versions of Solr without that line
-----Original Message-----
From: Oakley, Craig (NIH/NLM/NCBI) [C] via users <us...@solr.apache.org>
Sent: Thursday, December 4, 2025 11:05 PM
To: lamine lamine <ya...@yahoo.fr>; users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: RE: [EXTERNAL] Re: ZkCredentialsInjector issue
> Is your fille in.a location allowed by solr's security policy?
That was the relevant question
I had it in a directory next door to $SOLR_HOME (on a protected file system): a sibling directory from which we read our SOLR_ZK_CREDS_AND_ACLS: but of course we read SOLR_ZK_CREDS_AND_ACLS before we get into Java: and apparently the failure to read is coming from Java
I have moved the properties file into $SOLR_HOME itself; and now it works
Thanks
From: lamine lamine <ya...@yahoo.fr>
Sent: Thursday, December 4, 2025 8:14 PM
To: users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Is your fille in.a location allowed by solr's security policy?
Can you put it in "server/etc" like in the example?
DzkDigestCredentialsFile=
SOLR_HOME_DIR/server/etc/zookeepercredentials.properties"
On Thu, Dec 4, 2025 at 12:19 PM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk" (as previously mentioned)
SOLR_ZK_CREDS_AND_ACLS is
-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector -DzkDigestCredentialsFile=[the redacted filename]
From: lamine lamine <ya...@yahoo.fr>>
Sent: Thursday, December 4, 2025 12:27 PM
To: users@solr.apache.org<ma...@solr.apache.org>
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>>
Subject: [EXTERNAL] Re: ZkCredentialsInjector issue
The error says
: access denied ("java.io.FilePermission" "[file name redacted]" "read")
Can you check the permissions?
Can you as well share the config you're using for SOLR_ZK_CREDS_AND_ACLS
?
On Thu, Dec 4, 2025 at 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
For many years, we have been passing Zookeeper credentials through command line arguments (https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-system-properties): but it would be nice to hide the passwords away in a credentials file as described in https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-a-file
This succeeds when using "bin/solr zk", but not for the Solr instance itself. The solr.log file complains
2025-12-04 11:10:06.851 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsInjector: org.apache.solr.common.cloud.VMParamsZkCredentialsInjector
2025-12-04 11:10:06.852 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param ZkCredentialsInjector does not point to a class implementing ZkCredentialsInjector and with a non-arg constructor
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 61 more
2025-12-04 11:10:06.854 WARN (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
...
2025-12-04 11:10:06.970 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs
2025-12-04 11:10:06.982 ERROR (main) [ ] o.a.s.c.SolrCore null
org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.common.cloud.VMParamsZkCredentialsInjector'
at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:649) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
... 55 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 55 more
2025-12-04 11:10:06.990 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer is either not initialized or shutting down.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.c.SolrCore null
javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.CoreContainerProvider.waitForCoreContainer(CoreContainerProvider.java:154) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
2025-12-04 11:10:07.002 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@578524c3{solr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp}<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d>>>
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk"
Any suggestions what to try?
Thanks
Craig Oakley
Contract Worker (Other)
National Library of Medicine
National Institutes of Health
Building 45, room 6AS37D-57
301-496-6175
oakleycd@ncbi.nlm.nih.gov<ma...@ncbi.nlm.nih.gov>
<ma...@ncbi.nlm.nih.gov>
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Posted by "Oakley, Craig (NIH/NLM/NCBI) [C] via users" <us...@solr.apache.org>.
> Is your fille in.a location allowed by solr's security policy?
That was the relevant question
I had it in a directory next door to $SOLR_HOME (on a protected file system): a sibling directory from which we read our SOLR_ZK_CREDS_AND_ACLS: but of course we read SOLR_ZK_CREDS_AND_ACLS before we get into Java: and apparently the failure to read is coming from Java
I have moved the properties file into $SOLR_HOME itself; and now it works
Thanks
From: lamine lamine <ya...@yahoo.fr>
Sent: Thursday, December 4, 2025 8:14 PM
To: users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Is your fille in.a location allowed by solr's security policy?
Can you put it in "server/etc" like in the example?
DzkDigestCredentialsFile=
SOLR_HOME_DIR/server/etc/zookeepercredentials.properties"
On Thu, Dec 4, 2025 at 12:19 PM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk" (as previously mentioned)
SOLR_ZK_CREDS_AND_ACLS is
-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector -DzkDigestCredentialsFile=[the redacted filename]
From: lamine lamine <ya...@yahoo.fr>>
Sent: Thursday, December 4, 2025 12:27 PM
To: users@solr.apache.org<ma...@solr.apache.org>
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>>
Subject: [EXTERNAL] Re: ZkCredentialsInjector issue
The error says
: access denied ("java.io.FilePermission" "[file name redacted]" "read")
Can you check the permissions?
Can you as well share the config you're using for SOLR_ZK_CREDS_AND_ACLS
?
On Thu, Dec 4, 2025 at 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
For many years, we have been passing Zookeeper credentials through command line arguments (https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-system-properties): but it would be nice to hide the passwords away in a credentials file as described in https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-a-file
This succeeds when using "bin/solr zk", but not for the Solr instance itself. The solr.log file complains
2025-12-04 11:10:06.851 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsInjector: org.apache.solr.common.cloud.VMParamsZkCredentialsInjector
2025-12-04 11:10:06.852 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param ZkCredentialsInjector does not point to a class implementing ZkCredentialsInjector and with a non-arg constructor
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 61 more
2025-12-04 11:10:06.854 WARN (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
...
2025-12-04 11:10:06.970 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs
2025-12-04 11:10:06.982 ERROR (main) [ ] o.a.s.c.SolrCore null
org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.common.cloud.VMParamsZkCredentialsInjector'
at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:649) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
... 55 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 55 more
2025-12-04 11:10:06.990 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer is either not initialized or shutting down.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.c.SolrCore null
javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.CoreContainerProvider.waitForCoreContainer(CoreContainerProvider.java:154) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
2025-12-04 11:10:07.002 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@578524c3{solr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp}<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d>>>
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk"
Any suggestions what to try?
Thanks
Craig Oakley
Contract Worker (Other)
National Library of Medicine
National Institutes of Health
Building 45, room 6AS37D-57
301-496-6175
oakleycd@ncbi.nlm.nih.gov<ma...@ncbi.nlm.nih.gov>
<ma...@ncbi.nlm.nih.gov>
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.
RE: [EXTERNAL] Re: ZkCredentialsInjector issue
Posted by lamine lamine via users <us...@solr.apache.org>.
Is your fille in.a location allowed by solr's security policy?
Can you put it in "server/etc" like in the example?
DzkDigestCredentialsFile=SOLR_HOME_DIR/server/etc/zookeepercredentials.properties"
On Thu, Dec 4, 2025 at 12:19 PM, Oakley, Craig (NIH/NLM/NCBI) [C] via users<us...@solr.apache.org> wrote: I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk" (as previously mentioned)
SOLR_ZK_CREDS_AND_ACLS is
-DzkACLProvider=org.apache.solr.common.cloud.DigestZkACLProvider -DzkCredentialsProvider=org.apache.solr.common.cloud.DigestZkCredentialsProvider -DzkCredentialsInjector=org.apache.solr.common.cloud.VMParamsZkCredentialsInjector -DzkDigestCredentialsFile=[the redacted filename]
From: lamine lamine <ya...@yahoo.fr>
Sent: Thursday, December 4, 2025 12:27 PM
To: users@solr.apache.org
Cc: Oakley, Craig (NIH/NLM/NCBI) [C] <cr...@nih.gov>
Subject: [EXTERNAL] Re: ZkCredentialsInjector issue
The error says
: access denied ("java.io.FilePermission" "[file name redacted]" "read")
Can you check the permissions?
Can you as well share the config you're using for SOLR_ZK_CREDS_AND_ACLS
?
On Thu, Dec 4, 2025 at 11:08 AM, Oakley, Craig (NIH/NLM/NCBI) [C] via users
<us...@solr.apache.org>> wrote:
For many years, we have been passing Zookeeper credentials through command line arguments (https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-system-properties): but it would be nice to hide the passwords away in a credentials file as described in https://solr.apache.org/guide/solr/9_2/deployment-guide/zookeeper-access-control.html#through-a-file
This succeeds when using "bin/solr zk", but not for the Solr instance itself. The solr.log file complains
2025-12-04 11:10:06.851 INFO (main) [ ] o.a.s.c.c.SolrZkClient Using ZkCredentialsInjector: org.apache.solr.common.cloud.VMParamsZkCredentialsInjector
2025-12-04 11:10:06.852 WARN (main) [ ] o.a.s.c.c.SolrZkClient VM param ZkCredentialsInjector does not point to a class implementing ZkCredentialsInjector and with a non-arg constructor
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 61 more
2025-12-04 11:10:06.854 WARN (main) [ ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
...
2025-12-04 11:10:06.970 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Could not start Solr. Check solr/home property and the logs
2025-12-04 11:10:06.982 ERROR (main) [ ] o.a.s.c.SolrCore null
org.apache.solr.common.SolrException: Error instantiating class: 'org.apache.solr.common.cloud.VMParamsZkCredentialsInjector'
at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:649) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
...
... 55 more
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "[file name redacted]" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
...
... 55 more
2025-12-04 11:10:06.990 ERROR (main) [ ] o.a.s.s.CoreContainerProvider Error processing the request. CoreContainer is either not initialized or shutting down.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.s.SolrDispatchFilter Could not start Dispatch Filter.
2025-12-04 11:10:06.991 ERROR (main) [ ] o.a.s.c.SolrCore null
javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
at org.apache.solr.servlet.CoreContainerProvider.waitForCoreContainer(CoreContainerProvider.java:154) ~[solr-core-9.2.1.jar:9.2.1 a4c64ab6a2a270ca69c28c706dabb2927ed8a7c2 - jsweeney - 2023-04-24 11:35:31]
...
at org.eclipse.jetty.start.Main.main(Main.java:76) [start.jar:10.0.13]
2025-12-04 11:10:07.002 INFO (main) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@578524c3{solr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE}{/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp}<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d<mailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d%3cmailto:o.e.j.w.WebAppContext@578524c3%7bsolr-jetty-context.xml,/solr,file:///export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp/,AVAILABLE%7d%7b/export/home/dbman/solr/solr-9.2.1/server/solr-webapp/webapp%7d>>
I have confirmed that the account running the process does indeed have read access to the properties file (even granting write access does not help), and the properties file works for "bin/solr zk"
Any suggestions what to try?
Thanks
Craig Oakley
Contract Worker (Other)
National Library of Medicine
National Institutes of Health
Building 45, room 6AS37D-57
301-496-6175
oakleycd@ncbi.nlm.nih.gov<ma...@ncbi.nlm.nih.gov>
CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and are confident the content is safe.