Loading of zkCredentialsProvider has changed in Solr 7 or 8?

[Colvin Cowie <co...@gmail.com>]

Hello all

I hit another problem in moving from Solr 6 to 8.

We secure our ZooKeeper entirely (there's a restrictive ACL for every znode)
To pass the ZooKeeper credentials to Solr we implemented
ZkCredentialsProvider and ZkACLProvider to load the credentials from a file
on disk, which has the credentials in an encoded format (the details don't
really matter, just that we have our own implementations of the providers)

In Solr 6 we were able to specify the implementation classes in the
/solr.xml (zkCredentialsProvider & zkACLProvider) in the home directory for
each node and provided the implementation in a jar in the /lib directory of
the node, and this worked without issue.

In Solr 8.1.1, this still works for *some* of the requests that are made to
ZooKeeper, but other requests made by the SolrZkClient (via
SolrDispatchFilter) do not use the ZkCredentialsProvider specified in
solr.xml, instead falling back to the DefaultZkCredentialsProvider, or a
provider specified by the system property *-DzkCredentialsProvider.*

   - If Solr makes requests using the DefaultZkCredentialsProvider, they
   will fail with *NoAuth for /clusterprops.json.*
   - If I specify the *-DzkCredentialsProvider* then the jars in /lib are
   not visible to the SolrZkClient making the requests, resulting in a
   ClassNotFoundException (log attached).
   - If I add the jar directly to
   solr\server\solr-webapp\webapp\WEB-INF\lib then it is loadable, but I don't
   want to mess around with the lib directory Solr provides.
   - I know that the solrconfig.xml allows you to configure the <lib>
   directive for the classpath, but Solr won't be able to retrieve that
   solrconfig.xml from ZooKeeper without authenticating... Catch-22.


I've not had a chance to properly read the Solr 8 Reference Guide yet, but
it does still refer to the zkCredentialsProvider & zkACLProvider properties
being supported
https://lucene.apache.org/solr/guide/8_0/format-of-solr-xml.html#the-solr-element
I didn't see anything particular in the upgrade notes for Solr 7 or 8 about
changes to zkCredentialsProvider

So is there something wrong with what we were doing in Solr 6? Or has there
been a regression?

Thanks for any advice
Colvin

Re: Loading of zkCredentialsProvider has changed in Solr 7 or 8?

[Colvin Cowie <co...@gmail.com>]

I realize that attachments might not work on the mailing list, so here is
the log on Drive
https://drive.google.com/file/d/0B7mypFpwbHptWkp0X2U0azU2dGREb1k2WGlpeUM3MlRIWmRB/view?usp=sharing

On Tue, 11 Jun 2019 at 11:21, Colvin Cowie <co...@gmail.com>
wrote:

> Hello all
>
> I hit another problem in moving from Solr 6 to 8.
>
> We secure our ZooKeeper entirely (there's a restrictive ACL for every
> znode)
> To pass the ZooKeeper credentials to Solr we implemented
> ZkCredentialsProvider and ZkACLProvider to load the credentials from a file
> on disk, which has the credentials in an encoded format (the details don't
> really matter, just that we have our own implementations of the providers)
>
> In Solr 6 we were able to specify the implementation classes in the
> /solr.xml (zkCredentialsProvider & zkACLProvider) in the home directory for
> each node and provided the implementation in a jar in the /lib directory of
> the node, and this worked without issue.
>
> In Solr 8.1.1, this still works for *some* of the requests that are made
> to ZooKeeper, but other requests made by the SolrZkClient (via
> SolrDispatchFilter) do not use the ZkCredentialsProvider specified in
> solr.xml, instead falling back to the DefaultZkCredentialsProvider, or a
> provider specified by the system property *-DzkCredentialsProvider.*
>
>    - If Solr makes requests using the DefaultZkCredentialsProvider, they
>    will fail with *NoAuth for /clusterprops.json.*
>    - If I specify the *-DzkCredentialsProvider* then the jars in /lib are
>    not visible to the SolrZkClient making the requests, resulting in a
>    ClassNotFoundException (log attached).
>    - If I add the jar directly to
>    solr\server\solr-webapp\webapp\WEB-INF\lib then it is loadable, but I don't
>    want to mess around with the lib directory Solr provides.
>    - I know that the solrconfig.xml allows you to configure the <lib>
>    directive for the classpath, but Solr won't be able to retrieve that
>    solrconfig.xml from ZooKeeper without authenticating... Catch-22.
>
>
> I've not had a chance to properly read the Solr 8 Reference Guide yet, but
> it does still refer to the zkCredentialsProvider & zkACLProvider properties
> being supported
> https://lucene.apache.org/solr/guide/8_0/format-of-solr-xml.html#the-solr-element
> I didn't see anything particular in the upgrade notes for Solr 7 or 8
> about changes to zkCredentialsProvider
>
> So is there something wrong with what we were doing in Solr 6? Or has
> there been a regression?
>
> Thanks for any advice
> Colvin
>