You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Shawn Heisey <ap...@elyograg.org> on 2023/03/25 05:39:29 UTC

Problem creating a collection with auth enabled

With this security.json file in cloud mode, which allows all permission 
for the one user it has, creating a collection fails, either with SolrJ 
or in the admin UI.  This is the case for 9.1.1 or 9.3.0-SNAPSHOT.

{
"authentication":{
    "class":"solr.BasicAuthPlugin",
    "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= 
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
    "class":"solr.RuleBasedAuthorizationPlugin",
    "permissions":[{"name":"security-edit",
       "role":"admin"}],
    "user-role":{"solr":"admin"}
}}

The errors that get logged make no sense to me.

With 9.3.0-SNAPSHOT and auth enabled the admin UI "Add Collection" 
button does show a popup (but the add fails), but with 9.1.1 and auth 
enabled, clicking "Add Collection" does absolutely nothing.  Tried in 
Chrome, Firefox, and Edge.

Thanks,
Shawn


Re: Problem creating a collection with auth enabled

Posted by Jan Høydahl <ja...@cominvent.com>.
Glad it was resolved  :)

> 27. mar. 2023 kl. 21:42 skrev Shawn Heisey <ap...@elyograg.org>:
> 
> On 3/27/23 12:10, Jan Høydahl wrote:
>> I think I recall that if you copy-paste security.json from admin, you should leave out the {"v":9} part when re-uploading it to zk, else it may be invalid.
>> Do you have a step-by-step guide with Docker to provoke the OKI issue on a single node, single shard 9.2.0? Including collection creation commands etc?
> 
> I just tried a sequence of steps ... updating branch_9x to the latest, building a 9.3.0-SNAPSHOT docker image, running the container in cloud mode, enabling auth (using bin/solr auth), and creating a collection in the admin UI.  It failed to fail, which is a good thing.  I was able to create the collection with no problem.
> 
> So whatever the problem was, it has now been fixed.  Apologies for the noise, I really was running into problems, problems that started with being unable to create collections in a customer environment.
> 
> Thanks,
> Shawn


Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/27/23 12:10, Jan Høydahl wrote:
> I think I recall that if you copy-paste security.json from admin, you should leave out the {"v":9} part when re-uploading it to zk, else it may be invalid.
> Do you have a step-by-step guide with Docker to provoke the OKI issue on a single node, single shard 9.2.0? Including collection creation commands etc?

I just tried a sequence of steps ... updating branch_9x to the latest, 
building a 9.3.0-SNAPSHOT docker image, running the container in cloud 
mode, enabling auth (using bin/solr auth), and creating a collection in 
the admin UI.  It failed to fail, which is a good thing.  I was able to 
create the collection with no problem.

So whatever the problem was, it has now been fixed.  Apologies for the 
noise, I really was running into problems, problems that started with 
being unable to create collections in a customer environment.

Thanks,
Shawn

Re: Problem creating a collection with auth enabled

Posted by Jan Høydahl <ja...@cominvent.com>.
I think I recall that if you copy-paste security.json from admin, you should leave out the {"v":9} part when re-uploading it to zk, else it may be invalid.
Do you have a step-by-step guide with Docker to provoke the OKI issue on a single node, single shard 9.2.0? Including collection creation commands etc?

What exactly is broken with bin/solr auth in branch_9x? Is there a bug report for it?

Jan

> 27. mar. 2023 kl. 19:21 skrev Shawn Heisey <ap...@elyograg.org>:
> 
> On 3/27/23 04:49, Jan Høydahl wrote:
>> Can you try to reproduce on a clean 9.2.0 download? Please record each step you perform so it is easy to follow.
>> The exceptions you face is PKI auth struggling between two nodes in your cluster, no idea why that is. Also try without forwardCredentials=true and see if any change.
> 
> I have one node, not two.  It's a single-node setup with the embedded ZK.  I update that setup to the latest from branch_9x from time to time.
> 
> In 9.2, collection creation with auth enabled works.  I am having a problem with collection creation in 9.1.1 in one environment, but traced that to them having a newline at the beginning of solrconfig.xml, so I will be able to create collections once I get that fixed.  Apparently xerces cannot deal with ANYTHING at the beginning of the file other than <?xml ... which is annoying.
> 
> I checked 9.2 by starting a docker container with -DzkRun and running bin/solr auth in the container, then modified security in the admin UI to produce the security settings shown below.  The password for the solr user in this security.json is solr.
> 
> In 9.3.0-SNAPSHOT, when auth is enabled, collection creation fails. This security.json triggers the issue, giving that weird PKI error.
> 
> {
>  "authentication":{
>    "blockUnknown":true,
>    "class":"solr.BasicAuthPlugin",
>    "credentials":{"solr":"zDIhokl5uHoJRfYo0eWffIcWaZ8vGe0jBB/TwdHVIkE= fvkz1xO7zng/IN6VpjDYbcfsnT4wbZfX91ZKO0Fqrlg="}},
>  "authorization":{
>    "class":"solr.RuleBasedAuthorizationPlugin",
>    "permissions":[{
>        "name":"all",
>        "role":"admin",
>        "index":1}],
>    "user-role":{"solr":"admin"},
>    "":{"v":9}}}
> 
> I would try with the security.json created by bin/solr auth, but bin/solr auth does not work.
> 
> Thanks,
> Shawn


Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/27/23 04:49, Jan Høydahl wrote:
> Can you try to reproduce on a clean 9.2.0 download? Please record each step you perform so it is easy to follow.
> The exceptions you face is PKI auth struggling between two nodes in your cluster, no idea why that is. Also try without forwardCredentials=true and see if any change.

I have one node, not two.  It's a single-node setup with the embedded 
ZK.  I update that setup to the latest from branch_9x from time to time.

In 9.2, collection creation with auth enabled works.  I am having a 
problem with collection creation in 9.1.1 in one environment, but traced 
that to them having a newline at the beginning of solrconfig.xml, so I 
will be able to create collections once I get that fixed.  Apparently 
xerces cannot deal with ANYTHING at the beginning of the file other than 
<?xml ... which is annoying.

I checked 9.2 by starting a docker container with -DzkRun and running 
bin/solr auth in the container, then modified security in the admin UI 
to produce the security settings shown below.  The password for the solr 
user in this security.json is solr.

In 9.3.0-SNAPSHOT, when auth is enabled, collection creation fails. 
This security.json triggers the issue, giving that weird PKI error.

{
   "authentication":{
     "blockUnknown":true,
     "class":"solr.BasicAuthPlugin",
     "credentials":{"solr":"zDIhokl5uHoJRfYo0eWffIcWaZ8vGe0jBB/TwdHVIkE= 
fvkz1xO7zng/IN6VpjDYbcfsnT4wbZfX91ZKO0Fqrlg="}},
   "authorization":{
     "class":"solr.RuleBasedAuthorizationPlugin",
     "permissions":[{
         "name":"all",
         "role":"admin",
         "index":1}],
     "user-role":{"solr":"admin"},
     "":{"v":9}}}

I would try with the security.json created by bin/solr auth, but 
bin/solr auth does not work.

Thanks,
Shawn

Re: Problem creating a collection with auth enabled

Posted by Jan Høydahl <ja...@cominvent.com>.
Can you try to reproduce on a clean 9.2.0 download? Please record each step you perform so it is easy to follow.
The exceptions you face is PKI auth struggling between two nodes in your cluster, no idea why that is. Also try without forwardCredentials=true and see if any change.

Jan

> 27. mar. 2023 kl. 03:40 skrev Shawn Heisey <ap...@elyograg.org>:
> 
> On 3/26/2023 4:25 PM, Jan Høydahl wrote:
>> Hi,
>> This works in 9.2. Just tested.
>> You may need to force-reload the Admin UI (Shift-F5) to load the latest JS and CSS.
>> What happens is then, with your security.json is that the "Add collection" button is disabled and greyed out.
>> Contrary to what you say, in the security.json you provided, the "solr" user does not have the "all" permission, he only has the "security-edit" permission.
> 
> Oops.  That was the file that I uploaded, looks like I did not properly download it after I modified it in the UI to have the all permission and blockUnkown enabled.  I thought I had.
> 
> The admin UI issue is fixed in 9.2.
> 
> But even with the admin UI fixed, collection creation fails for me when auth is enabled.  I'm working in 9.3.0-SNAPSHOT.  I gave the user every permission available in the admin UI, and collection creation still fails.
> 
> Noted that "bin/solr auth" does not work in 9.3.0-SNAPSHOT.  I was hoping to attempt the creation with an auth file generated by the CLI script.
> 
> Here's the security.json I have attempted.  It's worth noting that creation also fails with blockUnknown disabled.
> 
> {
>  "authentication":{
>    "class":"solr.BasicAuthPlugin",
>    "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="},
>    "blockUnknown":true,
>    "":{"v":10},
>    "forwardCredentials":true},
>  "authorization":{
>    "class":"solr.RuleBasedAuthorizationPlugin",
>    "permissions":[{
>        "name":"all",
>        "role":["admin"],
>        "index":1}],
>    "user-role":{"solr":"admin"},
>    "":{"v":6}}}
> 
> This paste contains the logs generated by the collection creation attempt:
> 
> https://paste.elyograg.org/view/f29212ca
> 
> The very first error looks like Solr is trying to use noggit to parse HTML.  Which seems very weird.  I have no idea how enabling auth could generate that error.
> 
> If I remove security.json from zookeeper and do Add Collection again, it succeeds.
> 
> Thanks,
> Shawn


Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/26/2023 4:25 PM, Jan Høydahl wrote:
> Hi,
> 
> This works in 9.2. Just tested.
> You may need to force-reload the Admin UI (Shift-F5) to load the latest JS and CSS.
> What happens is then, with your security.json is that the "Add collection" button is disabled and greyed out.
> Contrary to what you say, in the security.json you provided, the "solr" user does not have the "all" permission, he only has the "security-edit" permission.

Oops.  That was the file that I uploaded, looks like I did not properly 
download it after I modified it in the UI to have the all permission and 
blockUnkown enabled.  I thought I had.

The admin UI issue is fixed in 9.2.

But even with the admin UI fixed, collection creation fails for me when 
auth is enabled.  I'm working in 9.3.0-SNAPSHOT.  I gave the user every 
permission available in the admin UI, and collection creation still fails.

Noted that "bin/solr auth" does not work in 9.3.0-SNAPSHOT.  I was 
hoping to attempt the creation with an auth file generated by the CLI 
script.

Here's the security.json I have attempted.  It's worth noting that 
creation also fails with blockUnknown disabled.

{
   "authentication":{
     "class":"solr.BasicAuthPlugin",
     "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= 
Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="},
     "blockUnknown":true,
     "":{"v":10},
     "forwardCredentials":true},
   "authorization":{
     "class":"solr.RuleBasedAuthorizationPlugin",
     "permissions":[{
         "name":"all",
         "role":["admin"],
         "index":1}],
     "user-role":{"solr":"admin"},
     "":{"v":6}}}

This paste contains the logs generated by the collection creation attempt:

https://paste.elyograg.org/view/f29212ca

The very first error looks like Solr is trying to use noggit to parse 
HTML.  Which seems very weird.  I have no idea how enabling auth could 
generate that error.

If I remove security.json from zookeeper and do Add Collection again, it 
succeeds.

Thanks,
Shawn

Re: Problem creating a collection with auth enabled

Posted by Jan Høydahl <ja...@cominvent.com>.
Hi,

This works in 9.2. Just tested.
You may need to force-reload the Admin UI (Shift-F5) to load the latest JS and CSS.
What happens is then, with your security.json is that the "Add collection" button is disabled and greyed out.
Contrary to what you say, in the security.json you provided, the "solr" user does not have the "all" permission, he only has the "security-edit" permission.

Jan

> 26. mar. 2023 kl. 18:51 skrev Shawn Heisey <el...@elyograg.org>:
> 
> On 3/25/23 18:28, Shawn Heisey wrote:
>> On 3/25/23 16:31, Shawn Heisey wrote:
>>> On 3/24/23 23:39, Shawn Heisey wrote:
>>>> With 9.3.0-SNAPSHOT and auth enabled the admin UI "Add Collection" button does show a popup (but the add fails), but with 9.1.1 and auth enabled, clicking "Add Collection" does absolutely nothing.  Tried in Chrome, Firefox, and Edge.
>>> 
>>> I figured out why clicking the "Add Collection" button does nothing.  I needed to have the "collection-admin-edit" permission.  But that should not be required, as I have the "all" permission.
>>> 
>>> That looks to me like a bug.
> 
> Thankfully that problem is fixed in 9.2, and I can see where the javascript code was corrected.
> 
> I have not been able to track down the actual problem creating collections, though.  I suspect that the fix will be similar to what was done in the javascript -- when checking permissions, check for the explicit permission that was passed, and also check for the all permission.
> 
> Do I have enough info about the problem for a Jira issue, or could the problem be something I am doing wrong?
> 
> Thanks,
> Shawn


Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <el...@elyograg.org>.
On 3/25/23 18:28, Shawn Heisey wrote:
> On 3/25/23 16:31, Shawn Heisey wrote:
>> On 3/24/23 23:39, Shawn Heisey wrote:
>>> With 9.3.0-SNAPSHOT and auth enabled the admin UI "Add Collection" 
>>> button does show a popup (but the add fails), but with 9.1.1 and auth 
>>> enabled, clicking "Add Collection" does absolutely nothing.  Tried in 
>>> Chrome, Firefox, and Edge.
>>
>> I figured out why clicking the "Add Collection" button does nothing.  
>> I needed to have the "collection-admin-edit" permission.  But that 
>> should not be required, as I have the "all" permission.
>>
>> That looks to me like a bug.

Thankfully that problem is fixed in 9.2, and I can see where the 
javascript code was corrected.

I have not been able to track down the actual problem creating 
collections, though.  I suspect that the fix will be similar to what was 
done in the javascript -- when checking permissions, check for the 
explicit permission that was passed, and also check for the all permission.

Do I have enough info about the problem for a Jira issue, or could the 
problem be something I am doing wrong?

Thanks,
Shawn

Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/25/23 16:31, Shawn Heisey wrote:
> On 3/24/23 23:39, Shawn Heisey wrote:
>> With 9.3.0-SNAPSHOT and auth enabled the admin UI "Add Collection" 
>> button does show a popup (but the add fails), but with 9.1.1 and auth 
>> enabled, clicking "Add Collection" does absolutely nothing.  Tried in 
>> Chrome, Firefox, and Edge.
> 
> I figured out why clicking the "Add Collection" button does nothing.  I 
> needed to have the "collection-admin-edit" permission.  But that should 
> not be required, as I have the "all" permission.
> 
> That looks to me like a bug.

I gave the user every single permission shown in the admin UI. 
Collection creation still fails.

https://paste.elyograg.org/view/6b0e0ccc

Thanks,
Shawn

Re: Problem creating a collection with auth enabled

Posted by Shawn Heisey <el...@elyograg.org>.
On 3/24/23 23:39, Shawn Heisey wrote:
> With 9.3.0-SNAPSHOT and auth enabled the admin UI "Add Collection" 
> button does show a popup (but the add fails), but with 9.1.1 and auth 
> enabled, clicking "Add Collection" does absolutely nothing.  Tried in 
> Chrome, Firefox, and Edge.

I figured out why clicking the "Add Collection" button does nothing.  I 
needed to have the "collection-admin-edit" permission.  But that should 
not be required, as I have the "all" permission.

That looks to me like a bug.

Thanks,
Shawn