You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@solr.apache.org by "epugh (via GitHub)" <gi...@apache.org> on 2024/11/02 12:02:00 UTC
[PR] SOLR-17540: Remove Hadoop Auth Module [solr]
epugh opened a new pull request, #2835:
URL: https://github.com/apache/solr/pull/2835
https://issues.apache.org/jira/browse/SOLR-17540
# Description
Remove Hadoop Auth
# Solution
no more Hadoop Auth
# Tests
Just removing things
# Tasks
- [ ] Look at `solr-tests.policy`
- [ ] Do we still need `useShortName` feature, maybe only supported by hadoop-auth?
- [ ] remove licenses
- [ ] update versions.lock
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2457312337
There are some more hadoop auth cleanup in the security.policy
* https://github.com/apache/solr/blob/main/solr/server/etc/security.policy#L134
* https://github.com/apache/solr/blob/main/gradle/testing/randomization/policies/solr-tests.policy#L103
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1855430558
##########
solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java:
##########
@@ -41,17 +40,11 @@ public void init(Map<String, Object> initInfo) {
String roleName = e.getKey();
usersVsRoles.put(roleName, Permission.readValueAsSet(map, roleName));
}
- useShortName =
- Boolean.parseBoolean(initInfo.getOrDefault("useShortName", Boolean.FALSE).toString());
Review Comment:
we have this "useShortName" feature, but it turns out it's ONLY ever supported by Kerberos. So it is dead code/unused feature with that...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1855235319
##########
gradle/testing/randomization/policies/solr-tests.policy:
##########
Review Comment:
solr-tests.policy and security.policy need to be kept in sync, mostly. It's annoying. So I see you touched one in one way and the other in another way.
##########
solr/solr-ref-guide/modules/deployment-guide/pages/solr-on-hdfs.adoc:
##########
@@ -210,41 +210,6 @@ NRTCachingDirectory max cache size.
+
Pass the location of HDFS client configuration files - needed for HDFS HA for example.
-=== Kerberos Authentication Settings
Review Comment:
as I mentioned, I think this can stay
##########
solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java:
##########
@@ -41,17 +40,11 @@ public void init(Map<String, Object> initInfo) {
String roleName = e.getKey();
usersVsRoles.put(roleName, Permission.readValueAsSet(map, roleName));
}
- useShortName =
- Boolean.parseBoolean(initInfo.getOrDefault("useShortName", Boolean.FALSE).toString());
Review Comment:
what's this about?
##########
solr/server/etc/security.policy:
##########
@@ -131,16 +131,6 @@ grant {
permission javax.management.MBeanServerPermission "releaseMBeanServer";
permission javax.management.MBeanTrustPermission "register";
- // needed by hadoop auth
- permission javax.security.auth.AuthPermission "getSubject";
- permission javax.security.auth.AuthPermission "modifyPrincipals";
- permission javax.security.auth.AuthPermission "doAs";
- permission javax.security.auth.AuthPermission "getLoginConfiguration";
- permission javax.security.auth.AuthPermission "setLoginConfiguration";
- permission javax.security.auth.AuthPermission "modifyPrivateCredentials";
- permission javax.security.auth.AuthPermission "modifyPublicCredentials";
- permission javax.security.auth.PrivateCredentialPermission "org.apache.hadoop.security.Credentials * \"*\"", "read";
-
// needed by hadoop security
permission java.security.SecurityPermission "putProviderProperty.SaslPlainServer";
permission java.security.SecurityPermission "insertProvider";
Review Comment:
can these be removed too?
(note: to test, edit solr-tests.policy, not this policy (which is production), and run tests)
##########
solr/modules/hdfs/src/java/org/apache/solr/hdfs/HdfsDirectoryFactory.java:
##########
Review Comment:
It's not clear that you actually needed to touch this at all. I think the the Hadoop Auth module and HDFS's Kerberos support are completely decoupled. WDYT @risdenk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "gus-asf (via GitHub)" <gi...@apache.org>.
gus-asf commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2494228698
> Would someone be willing to look at what is going on in SolrDispatchFilter? Around
>
> https://github.com/apache/solr/blob/620175a5626e69823b5aec8e20734a6352195000/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java#L336
> it appears that MAYBE there is an opportunity for some refactoring to simplify the flow... Especially since we specifically mention the Hadoop Auth as the reason for the extra complexity.. I do not understand this bit and would love another set of eyes.... I could also see a path to updating the very lengthy comments to say "This has complexity due to hadoop auth partially, and now that it is gone there may be an opportunity for improvement"...
I wrote that comment to memorialize several hours of digging I did back when I moved startup to a context listener. One of the things I found perplexing about SolrDispatchFilter when I first tried to understand it for that task was the lack of a call to doFilter(req,resp,filterchain) ... note that our custom version with the boolean retry doesn't count, because it doesn't make the normal call to the method specified by javax.servlet.Filter. Normally filter implementations look like:
````
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
// Do stuff, that seems important here
if (importantStuffSeemsHappy) {
chain.doFilter(request, response, chain);
} else {
// do unhappy error type stuff.
}
// add try/finally if there is mandatory cleanup.
}
}
````
So it was very weird not to find a call to doFilter in the doFilter method, nor in our custom version of it. EVENTUALLY I figured out that that call is made either in the dispatch method, OR in our auth filter (I haven't tried to prove it can't get called twice, but with just SolrDispatchFilter in play that is not currently going to cause a problem since chain.doFilter is a no-op for the final filter). One of the long term goals I have is to start pulling stuff that we are doing in this monster filter out int a series of filters, which will make the individual missions easier to understand and put the cleanup code near the instantiation code where, again it would be much easier to understand (and nesting can be easily seen to be correct).
My impulse (not yet informed by actual attempts) is to rework our auth plugin to be auth filters. The other thing I'm pointing out in that comment is that the HadoopAuthFilter is what seems to stand in the way of writing an if block such as:
````
if (authPlugin.authenticate(req, resp)) { // <<< note the lack of filterchain arg
// do searchy stuff here
chain.doFilter(request, response, chain);
} else {
// do unhappy 401 error type stuff.
}
// add try/finally for mandatory cleanups.
}
````
That is of course the first step to breaking auth out to it's own filter where it becomes
````
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
if (authPlugin.authenticate(req, resp)) {
chain.doFilter(request, response, chain); // <<< dispatch filter is later in the chain.
} else {
// do unhappy 401 error type stuff.
}
// add try/finally for mandatory cleanups.
}
````
The particular issue with the hadoop auth plugin that complicates the transition is that chain.doFilter() comes before a switch statement and other code...
https://github.com/apache/solr/blob/6f94c505fc923c452c58c2a3946ee4408ea06fac/solr/modules/hadoop-auth/src/java/org/apache/solr/security/hadoop/HadoopAuthPlugin.java#L247
At least at the time of that comment it seemed that all the other plugins called chain.doFilter() at the end (or possibly in a shortcut followed by an immediate return statement). Only Hadoop auth seemed to have mandatory actions AFTER doFilter(). If it disappears, we can possibly remove the filterchain argument and make a simpler use of the return value from authenticate().
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1864224898
##########
solr/modules/hdfs/src/java/org/apache/solr/hdfs/HdfsDirectoryFactory.java:
##########
@@ -16,8 +16,6 @@
*/
package org.apache.solr.hdfs;
-import static org.apache.hadoop.fs.CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION;
Review Comment:
will do! Aside from these changes, how does the rest look?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1830090073
##########
dev-tools/scripts/refguide/htaccess.txt:
##########
@@ -1,6 +1,6 @@
# Existing pages moved to sub path in the 9.0 guide
RedirectMatch 301 ^/guide/(about-this-guide|relevance|solr-glossary|solr-tutorial)\.html /guide/solr/latest/getting-started/$1.html
-RedirectMatch 301 ^/guide/(aliases|audit-logging|authentication-and-authorization-plugins|basic-authentication-plugin|circuit-breakers|client-apis|cloud-screens|cluster-node-management|collection-management|collections-core-admin|configuring-logging|enabling-ssl|hadoop-authentication-plugin|indexupgrader-tool|installing-solr|jvm-settings|jwt-authentication-plugin|kerberos-authentication-plugin|mbean-request-handler|metrics-reporting|performance-statistics-reference|ping|plugins-stats-screen|replica-management|rule-based-authorization-plugin|securing-solr|security-ui|shard-management|solr-control-script-reference|solrcloud-recoveries-and-write-tolerance|solrcloud-with-legacy-configuration-files|taking-solr-to-production|thread-dump|upgrading-a-solr-cluster|zookeeper-access-control|cert-authentication-plugin)\.html /guide/solr/latest/deployment-guide/$1.html
+RedirectMatch 301 ^/guide/(aliases|audit-logging|authentication-and-authorization-plugins|basic-authentication-plugin|circuit-breakers|client-apis|cloud-screens|cluster-node-management|collection-management|collections-core-admin|configuring-logging|enabling-ssl|indexupgrader-tool|installing-solr|jvm-settings|jwt-authentication-plugin|mbean-request-handler|metrics-reporting|performance-statistics-reference|ping|plugins-stats-screen|replica-management|rule-based-authorization-plugin|securing-solr|security-ui|shard-management|solr-control-script-reference|solrcloud-recoveries-and-write-tolerance|solrcloud-with-legacy-configuration-files|taking-solr-to-production|thread-dump|upgrading-a-solr-cluster|zookeeper-access-control|cert-authentication-plugin)\.html /guide/solr/latest/deployment-guide/$1.html
Review Comment:
i didn't try to runt he script, just updated the data...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2459554765
> @janhoy I tried running that script, but couldn't quite grok it. COuld you give an example of what that scripts should be? And let's add an example to the readme or the to script itself!
It's some time ago, and the script was made to make sure we had redirects for the old regfuide structure, and we also added in page removals. The gist is to mainain the csv file with metadata of all changes, and edit the py script to output the correct htacces.
We likely need to make another CSV section for pages removed in 10.0 guide, and then generate the correct redeirects..
I don't remember where / how the generated htaccess file is checked in though. And I know Antora has some built-in support for generating htaccess as well, should look into it..
To be pragmatic and unblock this, I'd make a Blocker JIRA for 10.0 and maintain a list of the removed pages there, so someone can update htaccess in proper way in due time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1851620976
##########
solr/modules/hadoop-auth/src/java/org/apache/solr/security/hadoop/KerberosPlugin.java:
##########
Review Comment:
I started thinking that there were TWO things... Kerberos in the Solr side, and then a seperate Kerberos in SolrJ capablity that somehow stays... I'm going to check what files I may have missed...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1855430319
##########
solr/solr-ref-guide/modules/deployment-guide/pages/solr-on-hdfs.adoc:
##########
@@ -210,41 +210,6 @@ NRTCachingDirectory max cache size.
+
Pass the location of HDFS client configuration files - needed for HDFS HA for example.
-=== Kerberos Authentication Settings
Review Comment:
Restored these changes.... I really don't have a sense one way or the other... If it's not correct, hopefully some of our Hadoop HDFS users will speak up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2488589191
> So Kerberos Auth is going away with this. Be sure to highlight this in CHANGES.txt and in JIRA. To me, Hadoop-Auth is a forgettable implementation detail (I didn't know _really_ know what this is) to the effective industry feature (I know of Kerberos). If you search our ref guide for "Kerberos", it still exists in this PR. See `authentication-and-authorization-plugins.adoc`.
@dsmiley I wanted to follow up on this. Is it possible that Kerberos still exists without Hadoop Auth? I assumed Hadoop Auth was different thatn Kerberos, in that with Kerberos you could authenticate in a Kerberos environment, and that didn't require Hadoop Auth.. That Hadoop Auth was a place to validate kerberous credentials, but you might validate elsewhere? There do seem to be lots of references to Kerberos that aren't tied to Hadoop Auth.. I think I need some guidence on where to go... Should I just remove the rest of the Kerberos code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2509521831
I think there might be a few more places to cleanup based on running the following on your branch
```
git grep -nFi kerber | grep -Fv -e 'solr/modules/hdfs' -e 'solr-on-hdfs.adoc' -e 'solr/CHANGES.txt' -e 'solr/benchmark/src/resources/words.txt'
```
Specifically these findings:
```
solr/bin/solr.in.sh:296:# Solr internally doesn't use cookies other than for modules such as Kerberos/Hadoop Auth. If you don't need any of those
solr/core/src/java/org/apache/solr/cli/AuthTool.java:124: + " bin/solr auth enable --type kerberos --config \\\"<kerberos configs>\\\" [--update-include-file-only <true|false>] [-v]\n"
solr/core/src/java/org/apache/solr/core/CoreContainer.java:602: // this caused plugins like KerberosPlugin to register its intercepts, but this intercept
solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java:328: // obviously don't care. Kerberos plugins seem to mostly use it to satisfy the api of a
solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java:351: // if it should short circuit, e.g. the Kerberos Authentication Filter will send an error and
solr/webapp/web/js/angular/controllers/index.js:27: // Needed for Kerberos, since this is the only place from where
solr/webapp/web/js/angular/controllers/index.js:28: // Kerberos username can be obtained.
```
It would be awesome to be able to cleanup the security policy files but I know there is some overlap with the Hadoop hdfs tests too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1864048605
##########
solr/modules/hdfs/src/java/org/apache/solr/hdfs/HdfsDirectoryFactory.java:
##########
Review Comment:
the modules/hdfs should not be touched. that is totally separate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2460047387
Got a little too frisky, broke something in the `hdfs` module:
```
2> 4749 ERROR (SUITE-HdfsCollectionsApiTest-seed#[54C70D1C3623A1D7]-worker) [n: c: s: r: x: t:] o.a.h.h.s.n.FSNamesystem FSNamesystem initialization failed.
2> => org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: Security Exception
2> at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2064)
2> org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: Security Exception
2> at org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2064) ~[hadoop-client-api-3.4.0.jar:?]
2> at org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:733) ~[hadoop-client-api-3.4.0.jar:?]
2> at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:683) ~[hadoop-client-api-3.4.0.jar:?]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2493669806
Would someone be willing to look at what is going on in SolrDispatchFilter? Around https://github.com/apache/solr/blob/620175a5626e69823b5aec8e20734a6352195000/solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java#L336 it appears that MAYBE there is an opportunity for some refactoring to simplify the flow... Especially since we specifically mention the Hadoop Auth as the reason for the extra complexity.. I do not understand this bit and would love another set of eyes.... I could also see a path to updating the very lengthy comments to say "This has complexity due to hadoop auth partially, and now that it is gone there may be an opportunity for improvement"...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2490177006
Hadoop Auth and Kerberos aren't the same. [Kerberos](https://en.wikipedia.org/wiki/Kerberos_(protocol)) is an authentication protocol, [Hadoop-Auth](https://hadoop.apache.org/docs/current/hadoop-auth/index.html) is framework that implements Kerberos and more. Hadoop-Auth seems to power several options, LDAP is another (I'm learning this now with some basic Google searches). @risdenk [wrote about it](https://risdenk.github.io/2018/11/20/apache-solr-hadoop-authentication-plugin-ldap.html) in 2018.
The underlying point is we should communicate the effective impact to users in terms they might understand. They might not know about the specific libraries that implement the functionality that are going away. Granted, a user using such stuff likely knows they are using this module since they had to opt-in to get the functionality (as of Solr 9 but not prior) but it may be an overlooked detail nonetheless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2490439421
I ran `find . -not -path '*/\.*' -type f -exec grep -l -i "kerberos" {} +` and wow... There are a LOT more files...
I think what I am hearing is that just removing the module and the light changes to the webapp isn't enough. There are a whole bunch of OTHER places that rely in one way or another on `hadoop-auth`? I had some thoughts of moving it to the sandbox, but if it's this ingrained in various locations in Solr, then that really doesn't make sense.
Does this align with other folks expectations that removing Hadoop Auth would touch all these files?
```
./solr/core/src/java/org/apache/solr/servlet/SolrDispatchFilter.java
./solr/core/src/java/org/apache/solr/core/CoreContainer.java
./solr/core/src/java/org/apache/solr/security/AuthorizationContext.java
./solr/core/src/java/org/apache/solr/cli/AuthTool.java
./solr/core/src/java/org/apache/solr/request/json/RequestUtil.java
./solr/bin/solr
./solr/bin/solr.cmd
./solr/bin/solr.in.sh
./solr/solrj/src/test/org/apache/solr/client/solrj/impl/BasicHttpSolrClientTest.java
./solr/solrj/src/java/org/apache/solr/client/solrj/impl/Krb5HttpClientBuilder.java
./solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpSolrClient.java
./solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-8.adoc
./solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-9.adoc
./solr/solr-ref-guide/modules/deployment-guide/pages/authentication-and-authorization-plugins.adoc
./solr/solr-ref-guide/modules/deployment-guide/pages/zookeeper-access-control.adoc
./solr/solr-ref-guide/modules/deployment-guide/pages/solr-on-hdfs.adoc
./solr/solr-ref-guide/modules/deployment-guide/pages/solr-control-script-reference.adoc
./solr/solr-ref-guide/modules/deployment-guide/pages/rule-based-authorization-plugin.adoc
./solr/solr-ref-guide/modules/getting-started/pages/solr-admin-ui.adoc
./solr/webapp/web/js/angular/controllers/index.js
./solr/server/etc/security.policy
./solr/CHANGES.txt
./solr/modules/hdfs/src/java/org/apache/solr/hdfs/HdfsDirectoryFactory.java
./gradle/documentation/render-javadoc/java11/package-list
./gradle/testing/randomization/policies/solr-tests.policy
./dev-tools/scripts/refguide/new-guide.txt
./dev-tools/scripts/refguide/old-guide.txt
./dev-tools/scripts/refguide/htaccess.txt
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2453055267
Kerb stuff appears to still work! All tests ran.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2495552427
Cause and effect is inverted. I suggest:
> Removed the Hadoop Auth module, and thus Kerberos authentication and other exotic options.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh merged PR #2835:
URL: https://github.com/apache/solr/pull/2835
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2459526827
@janhoy I tried running that script, but couldn't quite grok it. COuld you give an example of what that scripts should be? And let's add an example to the readme or the to script itself!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2511772758
Okay, I've responded (I think!) to @risdenk comments. I think this is ready for merging????
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2459891566
I added Priority=Blocker and FixVersion=10.0 to the JIRA to make it a valid 10.0 blocker.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1855437250
##########
solr/server/etc/security.policy:
##########
@@ -131,16 +131,6 @@ grant {
permission javax.management.MBeanServerPermission "releaseMBeanServer";
permission javax.management.MBeanTrustPermission "register";
- // needed by hadoop auth
- permission javax.security.auth.AuthPermission "getSubject";
- permission javax.security.auth.AuthPermission "modifyPrincipals";
- permission javax.security.auth.AuthPermission "doAs";
- permission javax.security.auth.AuthPermission "getLoginConfiguration";
- permission javax.security.auth.AuthPermission "setLoginConfiguration";
- permission javax.security.auth.AuthPermission "modifyPrivateCredentials";
- permission javax.security.auth.AuthPermission "modifyPublicCredentials";
- permission javax.security.auth.PrivateCredentialPermission "org.apache.hadoop.security.Credentials * \"*\"", "read";
-
// needed by hadoop security
permission java.security.SecurityPermission "putProviderProperty.SaslPlainServer";
permission java.security.SecurityPermission "insertProvider";
Review Comment:
okay, removing things broke things...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2510134527
@risdenk I see in `solr.in.sh` the reference to `-Dsolr.http.disableCookies=true` and that it highlights the use of it in either hadoop auth or maybe in a load balancer. Do you think i should remove that capablity? Or just remove the text referencing hadooop auth.. I worry that i'm going to pull on a thread and break the http clients... I could just edit it. If you have a sense of what the cookie change *shouild* be, please do push to the PR or add a patch and I'll add it... I'm thinking of just changing the text and leaving the rest of the cookie stuff... Maybe add another follow on JIRA...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2480560616
@malliaridis any chance you could look at this with me? I believe the `versions.lock` and `version.props` are actually the same because I copied from `main` over into my branch the files. I don't quite know how to tell it that `hadoop-auth/build.gradle` is actually gone~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2492127222
I'm not surprised there's a blast radius that exceeds the module itself. For everything outside the module, we'll need to review/edit with some care and it'll be on a case-by-case basis.
_For example_ HttpSolrClient was a surprising reference you made. I see one method withKerberosDelegationToken but it's only called by certain tests. Surprising to me; should have been documented as such and maybe not public. Furthermore the functionality itself doesn't *really* reference Kerberos; it references an abstract "delegation token" concept that perhaps might be meaningful outside Kerberos -- I really don't know about that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2466955710
So Kerberos Auth is going away with this. Be sure to highlight this in CHANGES.txt and in JIRA. To me, Hadoop-Auth is a forgettable implementation detail (I didn't know _really_ know what this is) to the effective industry feature (I know of Kerberos). If you search our ref guide for "Kerberos", it still exists in this PR. See `authentication-and-authorization-plugins.adoc`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2459640150
I have added https://issues.apache.org/jira/browse/SOLR-17546 to track the blocker task, I don't know if I did the "label" right, there were a lot to choose from!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2458257873
> There are some more hadoop auth cleanup in the security.policy
>
> * https://github.com/apache/solr/blob/main/solr/server/etc/security.policy#L134
>
> * https://github.com/apache/solr/blob/main/gradle/testing/randomization/policies/solr-tests.policy#L103
Thanks for that! I hope I got them all out...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "malliaridis (via GitHub)" <gi...@apache.org>.
malliaridis commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2480818067
With `hadoop-auth` gone, we have only one more blocker (s3) before migrating to jetty12. :D
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2495527115
I am going to not touch `SolrDispatchFilter` as I don't have a good game plan to move forward with it! Everything else is green.
For CHANGES.txt "Remove Kerberos authentication support from Solr. This in turn removes the Hadoop Auth module". <-- @dsmiley ???
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2459645491
Okay, I think I'm looking for an LGTM... I've also emailed the dev list with the PR.. No one has said "don't remove it".... Is that enought to click merge when I get an LGTM?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2466954428
Please do not merge this until after #2706 so we don't give Malliaridis a tough time over there. That one should be merged soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1851435460
##########
solr/modules/hadoop-auth/src/java/org/apache/solr/security/hadoop/KerberosPlugin.java:
##########
Review Comment:
Perhaps you deleted the whole module without examining its contents? This is KerberosPlugin, and its loss means no Kerberos with Solr.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1864048897
##########
solr/modules/hdfs/src/java/org/apache/solr/hdfs/HdfsDirectoryFactory.java:
##########
@@ -16,8 +16,6 @@
*/
package org.apache.solr.hdfs;
-import static org.apache.hadoop.fs.CommonConfigurationKeysPublic.HADOOP_SECURITY_AUTHENTICATION;
Review Comment:
These changes should be reverted. modules/hdfs is totally separate from hadoop-auth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2711845689
There are ~4 classes (+ 5 tests) containing "DelegationToken". @risdenk are these now obsolete/useless? Would that belong here in this issue (for Hadoop Auth) or would you recommend another issue? If another issue, can you please create it so that the wording is correct? These classes were added in [SOLR-9200](https://issues.apache.org/jira/browse/SOLR-9200)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1855422054
##########
solr/server/etc/security.policy:
##########
@@ -131,16 +131,6 @@ grant {
permission javax.management.MBeanServerPermission "releaseMBeanServer";
permission javax.management.MBeanTrustPermission "register";
- // needed by hadoop auth
- permission javax.security.auth.AuthPermission "getSubject";
- permission javax.security.auth.AuthPermission "modifyPrincipals";
- permission javax.security.auth.AuthPermission "doAs";
- permission javax.security.auth.AuthPermission "getLoginConfiguration";
- permission javax.security.auth.AuthPermission "setLoginConfiguration";
- permission javax.security.auth.AuthPermission "modifyPrivateCredentials";
- permission javax.security.auth.AuthPermission "modifyPublicCredentials";
- permission javax.security.auth.PrivateCredentialPermission "org.apache.hadoop.security.Credentials * \"*\"", "read";
-
// needed by hadoop security
permission java.security.SecurityPermission "putProviderProperty.SaslPlainServer";
permission java.security.SecurityPermission "insertProvider";
Review Comment:
okay, aligning the two files, and running tests..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2509523158
Some added context about delegation tokens - these were a Hadoop construct at one point and expanded elsewhere to avoid hitting the KDC (kerberos server) too much so the delegation token was used in place after the initial authentication happened. Basically it was a secure token passed around instead of doing the whole roundtrip to the KDC for each call. There are some other things the delegation token can do as well (impersonation if needed).
As David said the Hadoop authentication framework is not just Kerberos, but has a whole framework for authentication. Its similar to how Hadoop filesystem support isn't just HDFS but also S3 and some other backends.
Jetty does have Kerberos/SPNEGO support if we want to go down that route later. The Hadoop implementation for Kerberos support was better than most other Java support out there since not many Kerberos and Java implementations historically and lots of bugs across implementations (Active Directory vs Kerby vs others).
I do think its time to remove this module and make it fully opt in (via a plugin or separately supported module). I haven't had time to keep up with the Hadoop side development of this and don't use it anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "risdenk (via GitHub)" <gi...@apache.org>.
risdenk commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1864703595
##########
solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java:
##########
@@ -41,17 +40,11 @@ public void init(Map<String, Object> initInfo) {
String roleName = e.getKey();
usersVsRoles.put(roleName, Permission.readValueAsSet(map, roleName));
}
- useShortName =
- Boolean.parseBoolean(initInfo.getOrDefault("useShortName", Boolean.FALSE).toString());
Review Comment:
Yea the short name in Kerberos is without the realm. So if the principal was `USERNAME@REALM` it would just be `USERNAME` or if the principal was `USERNAME/HOST@REALM` it would just be `USERNAME` and is a Kerberos construct to break down the whole principal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "epugh (via GitHub)" <gi...@apache.org>.
epugh commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1830076740
##########
solr/webapp/web/partials/login.html:
##########
@@ -60,23 +60,6 @@ <h1>Basic Authentication</h1>
</div>
- <div ng-show="authScheme === 'Negotiate'">
- <h1>Kerberos Authentication</h1>
- <p>Your browser did not provide the required information to authenticate using Kerberos.
- Please check that your computer has a valid ticket for communicating with Solr,
- and that your browser is properly configured to provide that ticket when required.
- For more information, consult
- <a href="https://solr.apache.org/guide/solr/latest/deployment-guide/kerberos-authentication-plugin.html">
- Solr's Kerberos documentation
- </a>.
- </p>
- The response from the server was:
- <hr/>
- <pre>HTTP 401 {{statusText}}
-WWW-Authenticate: {{wwwAuthHeader}}</pre>
- <hr/>
- </div>
Review Comment:
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "janhoy (via GitHub)" <gi...@apache.org>.
janhoy commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1829415643
##########
solr/webapp/web/partials/login.html:
##########
@@ -60,23 +60,6 @@ <h1>Basic Authentication</h1>
</div>
- <div ng-show="authScheme === 'Negotiate'">
- <h1>Kerberos Authentication</h1>
- <p>Your browser did not provide the required information to authenticate using Kerberos.
- Please check that your computer has a valid ticket for communicating with Solr,
- and that your browser is properly configured to provide that ticket when required.
- For more information, consult
- <a href="https://solr.apache.org/guide/solr/latest/deployment-guide/kerberos-authentication-plugin.html">
- Solr's Kerberos documentation
- </a>.
- </p>
- The response from the server was:
- <hr/>
- <pre>HTTP 401 {{statusText}}
-WWW-Authenticate: {{wwwAuthHeader}}</pre>
- <hr/>
- </div>
Review Comment:
There is also traces of `Negotiate` authScheme in `login.js`
##########
dev-tools/scripts/refguide/htaccess.txt:
##########
@@ -1,6 +1,6 @@
# Existing pages moved to sub path in the 9.0 guide
RedirectMatch 301 ^/guide/(about-this-guide|relevance|solr-glossary|solr-tutorial)\.html /guide/solr/latest/getting-started/$1.html
-RedirectMatch 301 ^/guide/(aliases|audit-logging|authentication-and-authorization-plugins|basic-authentication-plugin|circuit-breakers|client-apis|cloud-screens|cluster-node-management|collection-management|collections-core-admin|configuring-logging|enabling-ssl|hadoop-authentication-plugin|indexupgrader-tool|installing-solr|jvm-settings|jwt-authentication-plugin|kerberos-authentication-plugin|mbean-request-handler|metrics-reporting|performance-statistics-reference|ping|plugins-stats-screen|replica-management|rule-based-authorization-plugin|securing-solr|security-ui|shard-management|solr-control-script-reference|solrcloud-recoveries-and-write-tolerance|solrcloud-with-legacy-configuration-files|taking-solr-to-production|thread-dump|upgrading-a-solr-cluster|zookeeper-access-control|cert-authentication-plugin)\.html /guide/solr/latest/deployment-guide/$1.html
+RedirectMatch 301 ^/guide/(aliases|audit-logging|authentication-and-authorization-plugins|basic-authentication-plugin|circuit-breakers|client-apis|cloud-screens|cluster-node-management|collection-management|collections-core-admin|configuring-logging|enabling-ssl|indexupgrader-tool|installing-solr|jvm-settings|jwt-authentication-plugin|mbean-request-handler|metrics-reporting|performance-statistics-reference|ping|plugins-stats-screen|replica-management|rule-based-authorization-plugin|securing-solr|security-ui|shard-management|solr-control-script-reference|solrcloud-recoveries-and-write-tolerance|solrcloud-with-legacy-configuration-files|taking-solr-to-production|thread-dump|upgrading-a-solr-cluster|zookeeper-access-control|cert-authentication-plugin)\.html /guide/solr/latest/deployment-guide/$1.html
Review Comment:
Instead of this, you should edit `mappings.csv` and `gen-refguide-redirects.py` similar to how the `_8_11` links for removed features in 9.x will then generate a `htaccess.txt` that redirect those pages to latest 8.x. So everything removed in 9.x should have the "latest" link redirect to a concrete 9.x version...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on code in PR #2835:
URL: https://github.com/apache/solr/pull/2835#discussion_r1851435460
##########
solr/modules/hadoop-auth/src/java/org/apache/solr/security/hadoop/KerberosPlugin.java:
##########
Review Comment:
I assumed you were aware of what you were removing, but maybe not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org
Re: [PR] SOLR-17540: Remove Hadoop Auth Module [solr]
Posted by "dsmiley (via GitHub)" <gi...@apache.org>.
dsmiley commented on PR #2835:
URL: https://github.com/apache/solr/pull/2835#issuecomment-2492129281
To answer some of these questions, it's helpful to do git-blame (I use my IDE actually) sometimes with history for the selected code to find the people/issues and then follow up with anyone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org